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ABSTRACT 


Non-developmental Items (NDI) acquisition programs are enjoying popular 
Support as faster, cheaper alternatives to full-scale development programs. 
Unfortunately, DOD policy with respect to risk management in NDI programs is 
lacking. Tailoring DOD risk management policy to support NDI program 
management leaves the program manager (PM) much guess-work. A NDI PM’s 
risk management program cannot reasonably benefit from DOD risk management 
guidance, procedures, and risk management tools because they are oriented to 
developmental program risks and risk management practices. Missing is any 
explicit consideration of the unique risks and risk management requirements in 
NDI programs. NDI PMs need more explicit guidance in policy and instruction 
regarding NDI risk management in the streamlined, accelerated NDI environment. 
This need is brought out in a case study of the Forward Area Air Defense Sensors 
Product Office which attempts to implement sound risk management into its NDI 
products without the benefit of definitive NDI risk identification, assessment, or 
response policy material. A lesson learned is the need for a published Risk 
Management Plan as the source of NDI risk management program decisions and 
actions. Specific recommendations are contained for inclusion in DOD policy with 


respect to NDI risk management. 
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I. INTRODUCTION 
A. BACKGROUND 


Today’s era of defense downsizing has the military 
services scrambling to protect research and development funds 
and to preserve their acquisition programs. Quality of life, 
training, sustainment, and readiness issues continue to siphon 
funds away from research, development, and acquiSition (RD&A) 
programs. The Services and their acquisition program managers 
(PMs) are hard-pressed to find ways to reduce development 
costs and program duration while optimizing new system 
performance and support capabilities. In these lean times, 
program management mistakes and problem-plagued acquisition 
strategies can often lead to program cancellation. Program 
risks must be carefully managed to prevent unanticipated or 
poorly planned-for events from making a program vulnerable to 
funding cuts or elimination. 

To the rescue, or so many hope, comes the Non- 
developmental Item (NDI). NDI system acquisitions give PMs 
and the Government greater access to technology. NDI 
acquisitions benefit industry by broadening the U.S. Defense 
Technology and Industrial Base (DTIB), increasing the number 
of defense contractors available to offer products to the 
Department of Defense (DOD), and providing more extensive 
Surge capability upon mobilization of the defense industry 
(Gansler, p. 110, 1994). Acquisition strategies based on 
these lower-cost, shorter-duration alternatives to the 
traditional, full-scale development programs offer the promise 
of effective risk mitigation and program success. NDIs 
seemingly offer the ideal, low-risk solution of state-of-the- 
art technological capabilities that can meet the Services’ 
urgent mission needs and operational requirements. 

NDI program management, however, is not without its own 


risks. A PM must tailor a program’s risk management to the 


unique risks and uncertainties in a NDI system acquisition. 
Tailoring is a practice that means modifying the acquisition 
process as necded to achieve favorable program results (DOD 
Instruction ({(DOBi) SOOU0.2 bp. 2-c lee The concept of 
tailoring risk management is valuable as part of the PM’s 
program authority. The problems arise in the implementation 
of tailored risk management programs, particularly when 
considering NDIs. 

Risk management in NDI acquisition programs requires more 
than tailoring. It demands policy, framework, and format to 
assist the program manager. The purpose of using a NDI 
acquisition strategy is to simplify and speed the acquisition 
process to meet a program’s, and ultimately, the soldier’s 
needs. NDI acquisition iS an approach that requires 
supportive risk management policy structure, rather than 
implicit expectations of tailoring, to fulfill its promise of 
Simple, rapid, and reduced-risk acquisitions. 


B. OBJECTIVE 


The objective of this thesis is to examine risk 
management in NDI acquisition programs. It will consist of 
DOD risk management and NDI policy reviews from an Army air 
defender’s point of view. Risk management concepts and 
practices will be explored first to set the stage for defense 
risk management. DOD risk management policies will be 
assessed to determine their applicability and support of NDI 
risk management programs. This thesis will then use the 
Ground Based Sensor (GBS) and the Light and Special Division 
Interim Sensor (LSDIS) acquisitions managed by the Forward 
Area Air Defense (FAAD) Sensors Program Office to demonstrate 
how risk management is currently practiced by an Air Defense 
Artillery (ADA) systems program office. The result of this 
examination will be recommendations for risk management 
policies and processes focused on NDI system acquisitions. 


C. RESEARCH QUESTIONS 


The primary research question of this thesis is: 
How does DOD risk management policy address NDI acquisitions 
and how does it support NDI acquisition programs such the 
Forward Area Air Defense Early Warning Sensors Program? 


Subsidiary Research Questions supporting this issue are: 

1. What is risk management? 

2. What are the current DOD and Army risk management 
policies and how do they address NDI acquisitions? 

3. What are the risks associated with NDI acquisitions? 


4. What were the risks involved with the GBS and LSDIS 
tactical early warning sensor acquisitions? 


5. What was the risk management approach for GBS and 
LSDIS and how consistent was the FAAD Sensor program’s 
risk management effort in complying with DOD and Army 


risk management policies? What problems were 
encountered in complying with risk management 
policies? 


6. What improvements can be made to DOD risk management 
policy to enhance NDI risk management planning and 
execution in future NDI acquisitions? 


ss 


D. SCOPE 


This thesis encompasses risk management policy as it 
applies to NDI acquisitions. It examines the current 
Structure of risk management in DOD and the U.S. Army. It 
considers the unique uncertainties and risk in NDI programs. 
It explores whether the intent of using NDI acquisitions as a 
risk management measure accomplishes its purpose. It focuses 
on the efforts of a NDI Air Defense system program management 
office (PMO) to manage its program and its program risks in an 
era of scarce resources and broad risk management policy. It 
is not meant to be a technical examination of the GBS or LSDIS 


systems capabilities nor a comprehensive evaluation of their 
acquisition strategies or plans. It will explore the 
treatment of aspects of risk within those documents but will 
not cover classified material. This research does not contain 
material from the Army’s risk management handbook (Department 
of the Army Pamphlet 70-2) scheduled to be published in 
February 1995. This examination assumes a working knowledge 


of a system acquisition life cycle. 
E. LITERATURE REVIEW AND METHODS 


The literature search to support this effort is driven by 
first-hand training experiences by the thesis author. The 
Starting point for this analysis comes from the Defense 
Systems Management College (DSMC) Risk Management, NDI 
Acquisition, Systems Engineering, and Commercial Practices 
publications, ADA Magazine articles on GBS and LSDIS, Program 
Manager Magazine articles on risk management, and the Air 
Force Acquisition Model (AFAM) materials relating to risk 
management. This is complemented by the Program Management 
Institute’s (PMI) guidelines for commercial program risk 
management which provide developed risk management theory. 
Journal articles and technical reportS examining risk 
management and NDI acquisitions were obtained from the Defense 
Logistics Systems Information Exchange (DLSIE) and the Defense 
Technical Information Center (DTIC). The FAAD Sensors Program 
Office, Assistant Program Executive Office for ADA 
Integration, and the Contracting Directorate at Redstone 
Arsenal, Alabama provided FAAD Sensors acquisition and risk 
management documents and data.* The TRADOC Systems Manager 
(TSM) for Early Warning Sensors and the Directorate for Combat 
Developments (DCD) at Ft. Bliss, Texas assisted by providing 
material on GBS and LSDIS fielding. 

The risk management materials for this research are 


augmented with face-to-face interviews with experts at 


Redstone Arsenal and Fort Bliss working with GBS and LSDIS 
acquisitions. Additional data came from telephone discussions 
with those same experts. The interviews solicited views and 
opinions on the GBS and LSDIS NDI acquisition strategies, 
their risk management programs, issues affecting NDI 
acquisitions, and their associated risks. Thesis travel 
occurred 14-18 June 1994 to conduct the personal interviews 
and discussions with these sources. 

The method this thesis employs is to familiarize the 
reader with risk management and NDI concepts and the DOD 
policies pertaining to them. It includes a review of existing 
risk management policies and procedures published and cited by 
defense sources. It progresses to a specific NDI program case 
analysis. DOD and Army risk management guidelines are 
examined through the case study of two ADA NDI acquisitions. 
The case reviews how DOD policy was implemented and to what 
extent the policy supports NDI acquisition program managers. 
The steps used to research and analyze this thesis topic 
consist of the following: 


a 


1. A familiarization and study of risk management 
concepts using the Program Management Institute’s body 
of knowledge pertaining to project risk management. 


2. An examination of defense risk management and risk 
Management policy. It is complemented by the DSMC 
publication on risk management. DOD Acquisition 
policy from the 5000.-series and Army risk management 
regulations provide sources for this step. 


3. An assessment of the support and effect of these 
policies on NDI program management using the DSMC 
publication on NDI acquistions and the above-mentioned 
references. 


4. An assessment of tailoring developmental risk 
management policy in support of NDI acquisition programs. 


S. An analysis of a case. The case examines the 
experience of the FAAD Sensors PMO using DOD risk 
management policies and practices in its program 
management of the GBS and LSDIS. The basis for this 
case is face-to-face and telephone interviews with 
past and present FAAD Sensors officials using a 
prepared set of questions on the FAAD Sensors program 
and risk management. 


6. Instruments of measurement for this thesis are 
templates from DOD Publication 4245.7-M, the risk 
assessment formats found in DOD 5000.2-M, and the risk 
management plan cited -in the DSMC risk management 
publication. A dendritic diagram will depict the 
differences in developmental item and NDI risk 
management policy. This may be valuable not only to 
the GBS and LSDIS case but for NDI acquisition 
programs overall. 


7. Based on case analysis and policy assessment, a series 
of recommendations and conclusions emerge to aid 
future NDI PMs. The usefulness of NDI-based risk 
templates and NDI-oriented risk management policy will 
be included as the basis for these recommendations. 


F. DEFINITIONS AND ACRONYMS 


Definitions and acronyms common to DOD, the U.S. Army, 
and civilian program management are noted throughout this 
thesis. 


G. CHAPTER OUTLINE 


This thesis investigates the use of risk management 
programs and policies for NDI acquisitions. 

Chapter I introduces the background and focus of 
research. It examines today’s constrained resources which 
complicate the acquisition and environment and support the 
appeal of NDI solutions. It considers the incentives for risk 


management and the unforgiving acquisition climate in which 


risk management has become a must. 

Chapter II serves as an overview to define and describe 
risk and risk management concepts. This risk management 
"primer", based on the PMI body of knowledge, then narrows to 
consider defense-unique details of risk management. 1G 6 
introduces current risk management policy at both DOD and Army 
level and lays out the general risk management process 
prescribed for program offices. 

Chapter III examines the NDI alternative to full scale 
development. It discusses the acquisition and risk management 
policy that is expected to support NDI system acquisition 
management. It includes consideration of the tailoring 
involved in NDI risk management. It examines the risks unique 
to pursuing NDI acquisitions. 

Chapter IV relates the events that created the current 
absence of Early Warning Network (EWN) systems (i.e. GBS and 
LSDIS) in ADA units. The risks in the GBS and LSDIS 
acquisitions may or may not be directly associated with the 
four year delay witnessed in providing initial sensor systems 
to the users. It provides an overview of the two sensors and 
considers the risks facing the GBS and LSDIS systems. 1c 
reviews the risk management approach for each sensor and 
presents challenges in mitigating NDI risks using 
developmental item-based policy, practices, and risk 
management tools such as the templates found in DOD 4245.7-M. 

Chapter V analyzes DOD NDI risk management policy and 
guidelines. This chapter considers the challenges and 
potential solutions for effective NDI risk management based on 
available risk management tools. The tools considered are the 
DOD 4245.7-M templates and a NDI risk management plan. 

Chapter val makes recommendations and proposes 
improvements in the content, value, and effectiveness of DOD 
NDI risk management policies and guidelines for future 


programs. These emerge from the application of lessons in the 


FAAD Sensors case and DOD policy review. The recommendations 
consist of revisions to risk management policies and practices 


in NDI acquisitions. 


II. RISK MANAGEMENT OVERVIEW 
A. PURPOSE 


This chapter introduces risk and risk management 


concepts. This risk management "primer" uses the Program 
Management Institute’s (PMI) body of knowledge on risk 
management as the source of this basic overview. The PMI 


approach is representative of the general body of knowledge 
concerning risk and risk management that may be found in 
business and industry today. DSMC risk management concepts 
are subsequently introduced. The chapter then presents DOD 
and U.S. Army risk management policy which acts as guidance 


for PMs as they structure their risk management programs. 
B. THE DEFINITION OF RISK 


What iS risk? Before answering this, it iS important to 
understand the role of uncertainty. Uncertainty represents 
the set of all possible outcomes that are either favorable 


fepporctunities) or unfavorable (risks) .? 


The goal of risk 
Management is to move uncertainties in an acquisition program 
away from the risk end of the uncertainty spectrum and towards 
the opportunities end. 

With this understanding of uncertainty, risk 1s defined 
as the cumulative probability of uncertain occurrences 
adversely affecting a program’s objectives. Risk itself 
consists of three risk factors. These are: 

a Risk Events. These are specific occurrences 
that can hurt progress in a program. ther iGonsequences. Of 
these negative events are described predominantly in terms of 
scope, quality, time, and cost objectives or goals: 


a. Scope Risks are changes in scope to meet 


1 The material presented in this chapter, sections B-F 
are referenced from Wideman, p.I-2 to p.B-1, 1992 except as 
noted. 


program objectives (e.g. regulatory or work breakdown 
structure changes). These risks include both ill-defined 
program pians and requirements. 

b. Quality Risks are performance or technical 
failures. These risks include poor quality assurance and poor 
workmanship. 

c. Schedule Risks are failures in meeting time 
estimates. These risks consist of poor, premature, or 
optimistic time estimates, scope changes, and/or accelerated 
fielding requirements. 

d. Cost Risks are failures to achieve desired 
budget targets. These risks consist of poor, premature, or 
optimistic cost estimates and/or inadequate cost controls. 

The type of acquisition program determines the type of 
possible risk events witnessed and their influence on a 
program. Most risks affect more than one program objective 
with the predominance of risk events affecting the schedule 
and cost program objectives. 

2. Risk Probability. This is the likelihood of 
occurrence of each of the risk events. 
3. Amount at Stake. This represents the severity 


of the consequences if a particular risk event occurs. 
C. RISK MANAGEMENT 


Risk management is "the art and science of identifying, 
assessing, and responding to project risk throughout the life 
of a project and in the best interests of its objectives" 
(Wideman, p. II-3, 1992). Risk management is a continuous, 
integrated process conducted throughout the life of an 
acquisition program. Its goals are the identification of 
program risk and the development of risk management strategies 
to minimize or avoid those risks. Risk management’s intent is 
to aid proactive decision-making through anticipatory planning 
aS opposed to crisis decisionmaking. 
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D. IMPLEMENTING RISK MANAGEMENT 


A program office implements risk management to assist 


planning and decisionmaking. In doing so, uncertainty, risk, 


and the amount at stake can be reduced to manageable levels 


@meoughout the program life. 
Risk management in business stresses four phases: 
ram 


identification, assessment, response, and documentation (see 


Figure 1 below). 
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Figure 1 The Four Phases of Risk Management (Source: 


Wideman, Chapter III, 1992) 
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1. Program Risk Identification 


Risk identification is the identification of specific 
characteristics thought to contribute to uncertainty in 
program objectives (Defense Advanced Research Projects Agency, 
D. 6, 1982)F, This first phase seeks to identify all the 
possible risks which may significantly impact a program’s 
success. Examples of sources of risk are: unanticipated 
Government intervention, unpredicted changes or shifts in 
economic markets, inappropriate program structure, scheduling 
delays, unanticipated resource problems, changes in 
technology, changes in performance requirements or design, 
increased technical complexity, and contractual difficulties 
evolving from the type of contract or data rights issues. 

Risk identification requires PMs to be familiar with the 
program’s scope, product, and objectives. Risks are matched 
to the program objectives they affect and then screened by 
their type and source. Identification should be completed 
early in a program's life to facilitate informed 
decisionmaking and provide risk early warning signals. PMs 
can identify sources of risk and understand them better using 
a multi-functional team to examine program details and 
determine their risks and uncertainties. These can then 


provide PMs initial risk estimates. 


2. Program Risk Assessment 


Risk assessment is the "examination of program elements, 
using experts, in order to synthesize a model...(it) is a 
means of reducing the knowledge of program elements to 
quantitative expressions of uncertainty" (DSMC, p. II-2, 
1983). This phase seeks to analyze the individual elements 


that create uncertainty or problems, determine the 
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relationships among those elements, and then formulate a model 
to study the problem (Ingalls, p.28, 1983). Quantification 
then follows to develop mathematical descriptions of those 
relationships posed in the model. 

Risk assessment consists of two primary steps: assessing 
and assigning probabilities to risk events and analyzing the 
consequences on the project. Assessment includes assigning 
probabilities to undesired or unfavorable events (risks) and 
examining their affect on program objectives. This again can 
effectively be done using a multifunctional team. A risk 
assessment team rates program risks by type, impact, and 
probability of occurrence using subjective or quantitative 
estimates. Risk ratings normally consist of low, medium, and 
high based on their severity of impact. Risk assessment 
treats risks as probabilities of unfavorable events that occur 
within the range of total success and total failure. This 
range establishes a risk baseline and the ability to benchmark 
a program’s risk status against program objectives over a 
program’s life. 

Quantitative risk assessment is called risk analysis. It 
feels and quantifies risk events and thelr impact on program 
objectives. Analysis concentrates primarily on medium- and 
high-rated program risks with the highest likelihood of 
occurrence. Contingency allowances, such as management 
reserves, cover risks with lower severity. Together, risk 
assessment and the risk analysis which supports it provide a 
risk profile or picture that the PM can use to make informed 
program decisions. 

Risk analysis requires the PM to estimate the degree of 
uncertainty expected early in a program. This may be done 
uSing probability distributions, probability tree diagrams, 
modeling, or sensitivity analysis (as opposed to simply using 
subjective assessment or "gut feeling"). Risk analysis 
modeling techniques that support this phase include 
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probability distribution analysis, the Delphi Method, Monte 
Carlo simulations, and decision theory. 

Risk analysis should accompany risk assessment early in 
a program’s life cycle and can be tailored to the program. It 
is an iterative, continuous cycle used to establish or update 
the risk baseline. Its value iS as an integral part of 
program management planning. 


3. Risk Response 


Risk response iS program manager actions and strategy 
initiated once risks have been identified and assessed aemm. 
phase establishes strategies to effectively handle risk. 
Responses include: 


1. Acceptance. Management takes no action after 
recognition (management responds by adjusting 


operations or modifying objectives). 


2. Avoidance. Management takes proactive steps, makes 
contingency plans, or selects lower-risk alternatives. 


3. Reduction/Minimization. Management uses alternative 
approaches such as workarounds or collection of data 
and information to facilitate contingency planning. 


4. Sharing. Management shares risk through joint 
agreements or contract agreements. 


5. Transfer. Management deflects risk via a contract 
clause, liability or loss insurance, or warranties. 


6. Absorption. Management relies on contingency 
allowances or reserves. 
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7. A combination of the above methods. 


Program managers can hold periodic reviews to monitor 


changes in program risks and update proposed responses. 


4. Risk Data Collection, Application, and 
Documentation 


This final risk management phase builds a reliable 
database to allow continued risk evaluations for current 
programs and as reference sources for future programs. Risk 
management data sources are established through historical 
databases, current databases, and post-program reviews and 
archives. These aid the estimation, comparison, updating, and 
information exchange needs of the program manager. Data- 
building facilitates and simplifies risk and program decisions 
particularly when multifunctional teaming and high turnover 
prevail in a program office. ; 

Risk management iS a continuous, iterative process 
conducted throughout the lifecycle of a system’s acquisition. 
Managing the uncertainties of a program and the consequences 
of unfavorable events is a difficult task. DOD faces its own 
acquisition risks. Its risk management approach is the focus 


of the next section. 


E. RISK MANAGEMENT IN THE DEPARTMENT OF DEFENSE 


1. Risk Definition 


DOD defines risk somewhat differently from the definition 
presented earlier. The DOD definition considers subjectivity 
and impacts made on program objectives rather than simply 
considering its impact on the whole program. It defines risk 


as "a subjective assessment made regarding the likelihood or 
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probability of not achieving a specific objective by the time 
established with the resources provided or requested" (DODI 
5000.2, ‘pe 625-15) oo Se Risk is characterized in the 
following ways:? 

a. Technical Risk. This is performance-based risk 
in the development of a new design for better performance or 
with new constraints on existing performance. It includes 
operational capability, reliability, and producibility. 
Examples relating to technical risk are development and design 
based on unproven technologies, requirements changes, and/or 
configuration changes. 

b. Supportability Risk. This is performance-based 
risk in fielding and providing maintenance support to new 
systems. It results from technical or programmatic risks. 
Examples include unforeseen training requirements, and/or 
failure to achieve reliability, availability, and 
maintainability requirements. 

(Technical and Supportability Risk appear to correspond 
to Performance Risk defined in Section B). 

c. Programmatic Risk. This is environment -based 
risk from the impact of events beyond a PM’s control. These 
include higher level decisions and/or turbulence introduced 
from outside the program (e.g. Congress) and resource changes 
that can influence program success. Additional examples are 
material and personnel shortages or turbulence, contractor 
instability, and/or funding changes. 

(Programmatic Risk appears to correspond to Scope Risk 
defined in Section B). 

d. Cost Risk. This is the risk of cost projections 
not being realistic or of the program not progressing 
efficiently to meet established cost objectives. This often 


* The material in sections E-H is referenced from DSMC’s 
Risk Management Concepts and Guidance, p.3-4 to p.C-7, 1989 
except where noted. 
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results from sensitivity to technical, Supportability, or 
programmatic risks. Examples are increased overhead rates and 
cost estimation errors. 

e. Schedule Risk. This is the risk of schedule 
estimates being too optimistic or of the program not 
developing effectively to achieve established baselines. This 
often results from sensitivity to technical, cost, 
Supportability, or programmatic risks. Examples include 
problems resulting from concurrent development and production 
and schedule estimation errors. 

According to the DSMC, technical, Supportability, and 
programmatic risks act as sources of risk while cost and 
schedule risks are indicators of those sources of unfavorable 
ames. these five types of risk act interdependently to 
affect a program’s performance. 


2. Risk Management 


DOD defines risk management as "a systematic approach to 
identifying, analyzing, and controlling areas or events with 
a potential for causing unwanted change. It is an integral 
part of the overall program management effort" (PM Notebook, 
fee 5-3, 1992). This definition differs from the business 
definition in that it stresses a systematic approach rather 
than "art and science". DODI 5000.2 adds that these actions 
are taken to bring risk "to an acceptable level in selected 
areas (e€.g. cost, schedule, technical, producibility, etc.) 
ema the total program (DODI 5000.2, p. 15-15, 1989). Risk 
Management identifies and evaluates the vulnerable areas of a 
program. Its purpose is to provide a means of comparing risk 
management performance to a standard and tracking risk-related 
information. 
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3. The Defense Risk Management Process 


A PM’s risk management process sets objectives and 
repeatedly assesses the program for obstacles preventing those 
objectives from being attained. The DOD risk management 
process is similar to the basic process described earlier. It 
requires that criteria or a set of standards with which to 
judge successful risk assessment and mitigation be established 
early in the program’s life. The DOD process consists of a 
different set of phases from the basic process. It consists 
of planning, assesSing, analyzing, and handling risk (see 
Figure 2). This structure, like the basic approach, considers 
both internal and external circumstances affecting a program’s 
objectives. It likewise allows the PM to formulate 
alternative courses of action and to make rational decisions 
on monitoring and controlling the outcomes of program events. 


The four defense risk management process components are: 


a. Risk Planning. This first phase defines the 
program’s present and future risks. This establishes the 


need, purpose, and requirements’for managing particular risks 
within a program to eliminate, minimize, or contain them. Risk 
identification in this phase gives a "plain English" 
description of risk events that can prevent program goals from 
being achieved. The program’s Work Breakdown Structure 1s a 
favorite tool to provide a risk identification, rating, and 
analysis structure. Risk rating schemes include lessons 
learned, expert interviews, baselines, templates, analogy 
comparisons, program document evaluations, technology 
evaluations, and contractor risk evaluations. This phase 
corresponds with the business Risk Identification phase. 

b. Risk Assessment. This is the examination of a 
program objective and assessment of those risk events posing 
potential problems. Risk ratings and a risk baseline similar 


to those described earlier give PMs a means of determining a 
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Figure 2 The DOD Risk Management Process (Source: DSMC Risk 
Management Concepts and Guidance, Chapter 5, 1989) 


program’s risk status. The combination of assessment and 
analysis shows points in a program where the consequences of 
risk are likely to occur, their severity, and what drives 
them. The DOD 4245.7-M risk templates are common DOD tools 
for assesSing program risk. Risk quantification begins in 
this phase. Whereas business groups risk assessment and risk 
analysis together, the defense risk management process 
separates risk assessment and risk analysis. 

c. Risk Analysis. Risk analysis is an examination 
of the changes in consequences caused by changes to risk input 


variables. Analytical and quantitative methods are used to 


BE, 


demonstrate the potential impacts of risk consequences on a 
program. These methods include simulations, network models 
(PERT, CPM), cumulative probability distributions, life cycle 
cost programs, and quick reaction modeling. These tools allow 
a PM to develop risk-handling options and techniques. 

Risk analysis provides valuable outputs. One example of 
these is the watchlist (PM Notebook, p.4.5-4, 1992). A 
watchlist is a Signalling mechanism for risk-based events or 
outcomes. The list consists of areas of risk, their expected 
impacts, and ways to detect and manage each prioritized risk 
area. It acts as a worksheet to track risk management 
progress and to show where the PM can make acceptable risk- 
program performance tradeoffs. 

ar Risk-Handling. Handling risk areas 
completes the DOD version of the risk management process. 
This is the formulation of management options to control or 
minimize areas of risk. DOD risk-handling measures are 
categorized as avoidance, control and prevention, assumption 
of consequences, transfer of responsibility, and knowledge and 
research. Risk control processes are the most common risk- 
handling measures. They consist of event-based, Government 
reviews and audits paralleled by contractor systematic 
analyses, reviews, and tests. Such continuous monitoring and 
corrective actions must be coordinated and integrated into 
program plans and documents. This phase corresponds to the 
business phases of Risk Response and Risk Documentation. 

The DOD risk management process is sequential and 
continuous much like the risk management process described 
earlier. Risk management evaluates what is achievable against 
what is planned; identifies risk areas using lessons learned, 
risk templates, work breakdown structures, and expert 
interviews; rates and quantifies risks; analyzes their 
severity of impact on program completion; and offers risk- 
handling options and tools. Once risk handling options are 
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implemented, the PM should document risk areas for future 
programs’ benefit. A means of documenting the PM’s risk 
management program is through the publication of a risk 


management plan. 
F. DOD RISK MANAGEMENT POLICIES 


Risk management policies for acquisition programs are 
described in the DOD 5000-series policies. The purpose of the 
5000-series iS to provide acquisition program policy, 
regulations, instructions, and format for PMs - to include 
their risk management programs. DOD Directive (DODD) 5000.1 
1s the first authority for risk management policy. According 
memwODD 5000.1: 


Program risks and risk management plans shall be 
explicitly assessed at each milestone decision 
point prior to granting approval to the next 
acquisition phase. This includes: early 
identification of critical parameters, technology 
demonstrations to reduce risk, test and evaluation 
to determine system maturity and identify risks, 
contractor risk plans and trade-offs to keep risk 
at acceptable levels. 


The Directive points out that risk assessment and 
reduction plans are required as part of the acquisition 
process to ensure sound program management decisions (DODD 
5000.1, p. 1-5, 1991). 

DODI 5000.2 follows DODD 5000.1 with more detailed 
guidance on managing risk. DODI 5000.2 Part 5B, published 1 
February 1991 states: 


A risk management program shall be established for 
each acquisition program to identify and control 
performance, cost, and schedule risks using the 
areas of risk identified in DOD 4245.7-M. The risk 
management program must include provisions for 
eliminating these risks or reducing them to 
acceptable levels. Industry participation in risk 
management is essential. 
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DOD 4245.7-M is a document containing risk management 
templates. These templates are tools for assessing and 
reducing program risks. 

DODI 5000.2 elaborates on risk management in Part 5B, 
Change 1, dated 26 February 1993. It states "risks, risk 
reduction measures, rationale, and assumptions made in 
assigning risk ratings, and alternative acquisition strategies 
will be explicitly assessed at each milestone decision point." 

Finally, the 5000.2-M reference for acquisition 
documentation and formats addresses the risk management area. 
DOD 5000.2-M indicates "The Integrated Program Summary (IPS) 
for each milestone, shall assess the risks with respect to the 
threat, technology, design, and engineering support, 
manufacturing, cost and schedule. The IPS shall summarize the 


actions being taken to mitigate those risks." 
G. U.S. ARMY RISK MANAGEMENT POLICIES 


U.S. Army leadership in and treatment of risk management 
in research, development, and acquisition programs began in 
1970. (Muldrow;, p.-326), 1985). In DA PAM 11-2, Research and 
Development Cost Guide for Army Material Systems (May 1976), 
paragraph 3.5, PMs are encouraged to manage risks by treating 
costs as ranges rather than point estimates. The Army 
Development Acquisition and Readiness Command (now known as 
the Army Material Command) Regulation, DARCOM-R 11-1 (April 
1976) on systems analysis, focuses extensively on risk 
analysis guidelines. Appendix D of this DARCOM-R prescribes 
its Decision Risk Analysis Guidelines for PMs as follows: 

1. a. Define the problem 

b. Establish the decisionmakers’ preferences for 
trade-offs between cost, schedule, and/or performance 

2. Establish the alternatives 

3. Define the events 

4. Collect the data 


Ze 


Determine the program risks 
Select the best alternatives 


Perform sensitivity analysis 


Oo wn on wm 


Communicate the results 


These steps are consistent with guidelines suggested in 
both the DSMC’s risk management publication and the PMI body 
of knowledge. 


H. SUMMARY 


Through the DODD 5000.1 and DODI 5000.2, the Defense 
Department provides risk management material and guidance. 
The 5000-series was intended to communicate risk management 
policy and guidelines for all PMs. 

The problem with DOD risk management policy is that 
policy and guidelines written for the Cold War defense 
environment was oriented toward full-scale, developmental 
programs and the five-phase acquisition cycle. NDI are the 
exception to these rules. Their risk management programs must 
be tailored because no explicit NDI risk management exists. 
The new dimension of risk management of NDI acquisitions, the 
DOD policies concerned, and the difficulties involved are the 
focus of the next chapter. 
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III. RISK MANAGEMENT IN NDI ACQUISITIONS 
A. PURPOSE AND BACKGROUND 


This chapter examines NDI acquisitions and the conditions 
that make NDI acquisition strategies attractive. It defines 
what NDIS are and are not, their advantages and disadvantages, 
and the acquisition policies governing them. This sets the 
stage to study how DOD risk management policies and practices 
address NDIS and support the PM managing a NDI acquisition 
program. 

Commercial business and DOD have new roles and face new 
trends compared to those of the Cold War status quo. During 
the Cold War, the defense establishment drove the pace of 
technological innovation. Several trends have since shaped 
new roles (Borrus, p.41, 1990). First, civilian technology no 
longer trails military technology as it did for several 
decades during the Cold War era. Next, commercial production 
features greater quality and efficiencies in flexible 
manufacturing, multiple Pmeauicie lines, just-in-time 
inventories, and lower volume production runs. Finally, many 
critical defense technologies have considerable overlap with 


3° The need to modernize and retain 


commercial technologies. 
state-of-the-art capabilities for military forces in an era of 
capped military spending supports the purchase of commercial 
products for military use. The primary means of acquiring and 


fielding these products is with NDIs. 
B. THE DEFINITION OF NDI 


The basic definition of NDI is "a broad term covering 
Material available from a variety of sources, with little or 


no development effort required by the government" (DSMC NDI 


3 The material in this chapter references the DSMC NDI 
Acquisition Guide, p.xiii to p.78, 1992 except as indicated. 
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Acquisition Guide, pla 3,92. NDIs are an option for 
fielding systems more quickly and cheaply than with a new, 
developmental item. NDI acquisition strategies look to 
commercial vendors and federal, state, local, and foreign 
governments as sources for the desired item. They also 
consider any commercial item in design, development, and/or 
production but not yet available for commercial use (Prueitt, 
1994). 
The U.S. Congress defines a NDI as: 


1. Any item available in the commercial marketplace. 


2. Any previously developed item in use by the U.S. 
Government or cooperating foreign governments. 


3. Any item of supply needing only minor modifications to 
meet DOD requirements. 


The military services define NDIs more specifically. The 
Army version distinguishes NDI in three categories (Quindlen, 
p. 53, 1989): 


1. Category A (Basic NDI): Off-the-shelf items to be 
used in the same environment for which items were 
designed with little or no development or modification 
required. 


2. Category B (NDI Adaptation): Off-the-shelf items to be 


used in an environment different than that for which 
the items were designed with some development 
required. These include products requiring 
hardening, strengthening and/or related modifications. 
This category is sometimes referred to as "Ruggedized 
NDE 


3. Category C (NDI Integration): Integration of existing 
componentry and the essential engineering effort to 
accomplish systems integration with research and 
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development to integrate systems. This category is 
sometimes labelled "Militarized NDI". 


Confusion sometimes arises in distinguishing a NDI from 
a commercial-off-the-shelf (COTS) item. A COTS item makes up 
only a portion of DOD-classified NDIs. COTS items are 
commercial hardware/software items not modified by the 
government, items that are in the commercial inventory or 
production, that have proved their performance in a similar 
environment, that have an existing support structure, have an 
internal support structure, have an internal configuration 
that flows with commercial changes and generally are 
integrated with other hardware/software items to become part 


of a system or subsystem Gaeao st lee 
C. USING NDI FOR REDUCING PROGRAM RISKS 


NDIs have become increasingly important in equipping our 
services. NDI acquisitions are more prevalent as an 
alternative to costly, lengthy, and more risky developmental 
programs. The lower risks commonly associated with cost, 
schedule, and technical performance make NDI acquisitions 
politically and programmatically popular. 

NDIs are considered a means of mitigating program risks. 
Risk “mitigation involves the assessment and response 
activities of risk management which were outlined in the 
previous chapter. Risk mitigation is implemented through the 
shortened or eliminated Demonstration and Validation (DEMVAL) 
Phase and Engineering and Manufacturing Development (EMD) 
Phase of the NDI acquisition cycle. The streamlined NDI 
acquisition cycle reduces cost and schedule risks. However, 
NDI acquisitions do not eliminate risk but may in fact 
introduce new, significant risks. These risks will be 
examined shortly. 

An effective NDI acquisition program must fulfill the 
Service’s needs by fielding mature technology that preserves 
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combat readiness while satisfying popular expectations. The 
three features which attract substantial DOD and Congressional 
popular support to NDI alternatives but which challenge the PM 


are: 


alee Meeting user requirements based on available 
commercial market solutions. 

2. Tailoring user requirements to suit NDI acquisition 
strategies while avoiding "goldplating". 

ae Opening up greater access to state-of-the-art 
technology for DOD while keeping pace with changing threats, 
emerging technologies, and innovative combat systems. 

To DOD and Congressional decisionmakers, NDIS appear to 
be a low-risk approach for meeting the Services’ needs and 
requirements. The danger, however, is in forcing NDI 
solutions on PMs and their customers (the user community), 
when those NDI neither meet requirements nor transition 
effectively from commercial to battlefield environments. 
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D. NDI BENEFITS AND POTENTIAL RISKS 


1. Benefits 


NDI acquisitions have many benefits which make the 
transition of commercial technology to military use a popular 
course of action. A NDI acquisition provides an existing or 
modified system to meet a stated requirement in a cost- 
effective and timely manner. The benefits NDIs offer are: 


a. Time 
Decreased acquisition cycles, shorter procurement 
lead times, and reduced testing requirements often result in 


quicker system delivery and fielding (Cain, p. 22, 1993). See 
Figure 3. 
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b. Cost 


Minimized research and development costs reduce 
expected lifecycle costs. These costs savings stem from using 
commercial specifications and the competitive pricing inherent 


in commercial markets (Cain, p. 23, 1993) 
c. Technical Performance 


State-of-the-art technology from the commercial 
marketplace meets operational requirements with limited design 
and development. NDI programs introduce the acquisition of 
mature technologies, with validated and established production 
techniques, and high quality standards. Reliability, 
availability, and maintainability (RAM), and supportability 
data are already established as part of commercial testing and 


market demands (Garcia, p. 8, 1991). 
d. Risk Reduction 


Risks to development costs, program duration, and 
leading edge technical performance are decreased. NDI limits 
risks with an established cost structure, minimized production 
Start-up costs with Guigoneiy) Operating production lines, 
economies of scale where the Government is not the sole 
customer, and the competitive pressures of commercial markets 
On reasonably-priced, high volume production (Vandeviere, p. 
ms, 1994). 


e. Contracting 
Simplified contracts often result from procurement 


of established products. The increased use of Fixed-Price- 


type contracts puts more risk responsibility on the 
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Figure 3 Notional Developmental and Non-developmental Item 
Timelines (Source: Vandeviere, Chapter 2, 1994) 


contractors’ shoulders while reducing Government risk (GAO, p. 
a2) LOSS 


2. Potential Risks 


A developmental item PM traditionally faces cost, 
schedule, performance, supportability, and programmatic risks 
whereas the NDI PM’s program risks are more unique. NDIs 
typically reduce cost and schedule risks. The risks described 
below affect the technical performance, supportability, and 
programmatic objectives in a NDI program. In reviewing the 
literature, the author has determined that the following five 
areas are the most pressing potential risks facing NDI 
acquisition programs: Requirements, NDI Acquisition Management 
Environment, Specifications and Standards, Test and 
Evaluation, and Integrated Logistics Support. If not properly 
addressed, these risks may easily affect cost and schedule 
objectives as well. 
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a. Requirements 


"Initial (Operational) Requirements Documents (ORD) 
are developed without careful consideration of what is 
currently available as NDI" (DSMC NDI Acquisition Guide, p. 
fae 22). Requirements risks. surface when the operational 
community develops an ORD focused entirely on operational 
need, independent of market solutions available to meet those 


needs. NDI requirements risk can arise from users developing 





an ORD based on available commercial technology rather than 
available commercial products. The user proceeds E Onl 126 and. 
match the desired functions or capabilities of several 
products into one ORD when no existing item can meet user 
requirements without making extensive modifications. On the 
other hand, the user may accept an available NDI solution 
which does not strictly comply with all the specified 
requirements. 

Requirements risk affects all of a program’s 
objectives. This risk in NDI acquisitions often indicates a 
lack of flexibility in ORD development. The risk grows when 
the requirements generation process does not include the PM. 
The PM’s responsibility, as the material developer, is to 
respond to user needs but to be fully conscious of technical 
risks, affordability constraints, and schedule impacts. The 
user’S responsibility is to be realistic in stated 
requirements, weigh proven capability and rapid deployment 
benefits against performance limitations, and negotiate 
acceptable trade-offs with the PM. Too often these 
responsibilities are not integrated or communicated. 
Requirements risk affects a user’s desire or ability to back 
Off of validated ORD requirements. When minor modifications 
are pursued to comply with stated requirements, the NDI may 
take on extensive and costly developmental item 
characteristics and defeat the purpose of the NDI acquisition 
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strategy (Cain, p.24, 1993). 

Insufficient or ineffective market analysis 
accentuates the problem. Market analysis may not be detailed 
enough to identify technologies and potential contractors. 
The user/PM team then cannot evaluate the strengths of a NDI 
acquisition strategy compared to a developmental one. Market 
analysis, when properly used, contains both broadly-scoped 
market surveillance to canvas technologies and product 
developments and the narrower-focused market investigation 
used to determine what technology or products meet user needs. 
Insufficient commitment of effort and resources to teaming and 
market analysis leads the user and PM to either neglect 
potential alternatives or falsely conclude that no commercial 


alternative exists to fulfill user requirements. 


b. NDI Acquisition Management Environment 


The NDI acquisition management environment confronts 
the PM with the complexities of multiple stakeholders (DOD, 
Congress, the Service, and the user), internal and external 
perceptions and attitudes (cultural resistance) towards NDIs, 
commercial standards and practices, developmentally-based 
paperwork requirements, audits, and inspections, and mistrust 
among the NDI acquisition participants. These elements create 
programmatic risks for the PM. 

The successful achievement of cost, schedule, 
performance, and supportability objectives depends on the PM’s 
interaction and effective teaming with the prime contractor as 
well as with the user community. Trade-offs between ORD-based 
requirements and cost and schedule constraints on the program 
may become sources of conflict. The Government’s desire to 
buy aS opposed to the contractor’s unwillingness to provide 
adequate technical data may become a major source of friction. 


The contractor’s unwillingness to allow quality control 
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oversight and qualification of its manufacturing and 
production processes may contribute to a contentious 
relationship with the PM and the Government. NDI performance 
Seecctations May not be realistic to both parties. Lack of 
communication before and after program reviewS may prevent 
clearly-defined agreements and changes from being implemented 
during the NDI acquisition cycle. With the shortened NDI 
acquisition cycles, the communication and feedback loops may 
not develop and mature. The results may become mistrust and 
inappropriate trade-off decisions. 

NDI acquisition success requires changing 
traditional, sometimes antagonistic procurement mindsets in 
the acquisition and user communities. The NDI environment may 
not be conducive to effective communication, coordination, and 
cooperation. The willingness to objectively assess and choose 
between the best technology available and a full-scale 
Gevelopment start-up requires open-minded acquisition team 
members. The team members’ responsiveness to new acquisition 
strategies requires overcoming ingrained developmental program 
biases. Such an atmosphere may require extensive training and 
education (Garcia, p. 26, 1991). 

Political and defense interests may drive an 
acquisition towards a NDI acquisition strategy when it is not 
appropriate. This confines the PM’s efforts to the "NDI box 
(with) NDI boundaries and NDI criteria" when that NDI option 
may not meet the user’s requirements. The appeal of NDIS may 
foster "zero defects" expectations whereby few, if any, 
program or contractual problems will be accepted, especially 
if the user has experienced previous acquisition program 
failures. Given constrained budget and resource environments, 
programming and budgeting decisions from outside the PM’s 
sphere of influence may create programmatic risks themselves. 

Lastly, NDI streamlining may ignore useful development 


program management tools such as program reviews and design 
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reviews. These tools may be ignored to meet accelerated 
schedules and to minimize or streamline the commitment of 
resources, funds, and personnel. 

c. Specifications and Standards 


Specifications and standards serve as a means of 
requirements verification in a product’s design and 
development but they contribute to technical performance risks 
in a program. The risk in specifications and standards (S&S) 
concerns the use of military as opposed to commercial S&S. 

Military specifications (MILSPECS) and military 
Standards (MILSTDS) draw criticism from industry as being 
excessively demanding, outdated, incompatible with NDIs, and 
unnecessary in light of the detail presently existing in 
commercial item descriptions (CIDs). In some cases such as 
nuclear delivery systems, strict MILSPECS and MILSTDS are 
justifiable, but the tendency is to go beyond performance 
(form, fit, and function) requirements to describe the design 
and manufacturing processes themselves. The S&S risks 
materialize in contract prices, timeliness, access, and 
manufacturing efficiency, all of which may be lost to the 
contractor and the user if MILSPECS and MILSTDS are required 
in product design, development, and manufacturing. 

The challenge for NDI PMs is to leave the safety and 
security of MILSPECS and MILSTDS and to use CIDs and 
commercial standards. A NDI acquisition strategy encourages 
acquisition agencies to break inclinations to "cling to the 
security of MILSPECS and MILSTDS" and instead to take 
advantage of commercial technology development, innovation, 
and streamlining initiatives. The PM’s risk is one of 
accepting and using unfamiliar commercial design and 
development parameters. With commercial S&S come assumptions 
about the conditions and tasks under which the product must 
perform as well as the intended purpose of the standard. The 
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S&S may be unique to a particular company, product, or process 
and not easily transferrable. 


The specifications and standards risks consist of 
either choking the NDI process with required MILSPECS or 
relying on CIDs for products, parts, and spares which may or 
may not match MILSPEC and MILSTD detail. The implicit 
understanding of combat environments and conditions may also 
be lost. The grasp of military operational conditions in 
commercially-oriented S&S may be foregone. Competitive 
reprocurements that depend on CIDs as technical data package 
references may jeopardize functional and technical performance 
in the field. This in turn depends on the PM being equipped 
with a detailed market analysis that examines the suitability 
of CIDs to meet user and system requirements. 

A shrinking DTIB is one trend DOD witnessed after 
winning the Cold War. Stemming this trend is potential dual- 
use (military and civilian) item applications. Defense- 
commercial product usage enables more flexibility in 
procurements, closes the gap between military and commercial 
products, and keeps the DTIB competitive in the global 
marketplace (Gansler, p. 115, 1993). Yet the defense 
industry, like the acquisition workforce, is resistant to 
these opportunities (Mandel, p. 193, 1994). Dual-use and 
commercial specifications in this post-Cold War era are 
becoming more suitable for meeting the level of detail 
required for military requirements. But despite their 
political popularity, commercial market practices, 
Specifications, and production standards continue to be a hard 
sell (Schrage, p. B3, 1994). 
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ad. Test and Evaluation 


Test. and evaluation (T&E) programs for NDIs must 
achieve the same purpose as developmental T&E programs: to 
define risks. This risk functions to consider how a system 
meets its functional specifications in the intended 
operational environment. The T&E risks in NDI acquisitions 
are: testing away NDI time and cost savings for the sake of 
testing and risk elimination; incomplete or inadequate design, 
development, production, and logistics support T&E by both the 
contractor and PM due to the pressures of an accelerated 
acquisition (Cain, p. 24, 1993); or assuming the contractor’s 


T&E results are sufficient and merely conducting technical 


reviews Of T&E data (Garcia,  p.n20, melo Ineffective 
verification of compliance with requirements and 
specifications may result. For example, prototypes may be 


proposed as competitive designs without sufficient production 
and support facilities and activities. PMs who implement 
streamlined NDI T&E programs and use T&E data reviews in place 
of production and support capability evaluations (or at least 
demonstrations) may be creating risks in their programs. NDI 
T&E risks therefore, can contribute to technical performance 
and supportability risk in the program. 

NDI T&E programs are a form of risk management, not 
risk elimination, used to prove technical capabilities, 
operational effectiveness, and suitability. The T&E community 
sees four areas of risk in NDI system testing: 


1. Requirements may not be clearly understood. 


2. Definition of the operational environment may not be 
Clear and thus can’t be tested with a level of 


confidence. 
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3. Definition of system interfaces and operability 
between the NDI system and others it must 


interface/operate with may not be clear. 


4. Test documentation and support issues may not be 
addressed in the test program. 


Two impediments in the T&E community also exist: 


1. The T&E "mindset". The testers orient on traditional 
developmental approaches and ignore non-traditional, 
NDI T&E approaches tailored to a streamlined 
acquisition strategy. This non-traditional approach 
may require different .T&E methods, timelines, and 
levels of detail. 


2. Lack of knowledge and experience with commercial test 
practices and standards. 


A streamlined NDI T&E program may introduce risks of 
insufficient source evaluation, inadequate competition, and 
incomplete market analysis. This may lead the PM to overlook 


promising alternatives. 


e. Integrated Logistics Support 


"ILS in an NDI acquisition can be more difficult 
because of the difference in an acquisition process" (DSMC NDI 
Acquisition Guide, p. 41, 1992). NDI integrated logistics 
support (ILS) activities performed in DEMVAL and EMD are often 
accelerated to ensure support arrives concurrently with the 
item. ILS risks involve events that disrupt maintenance 
planning, manpower, supply support, support equipment, 
technical data, training and training support, mission 
critical computer requirements, packaging/handling/storage/ 
transportation, facilities, and design interface of 


operational capability with resources. The ILS issue becomes 
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one of identifying what trade-offs, logistics-related 
requirements, and departures from developmental ILS are 
necessary. 

A NDI acquisition strategy creates several ILS 
risks. While real reliability and training data may be in 
place, reduced lead time means less ILS planning time and time 
to develop organic support. The ILS risk of proliferating 
hardware and software changes (e.g. when DOD is unable to buy 
the technical data rights for maintaining configuration 
control of commercial items) can be significant. ILS risks can 
contribute tremendous complications to NDI acquisition 
planning and execution. 

A PM develops an ILS plan (ILSP) in conjunction with 
the NDI acquisition strategy. This outlines risk management 
measures for supportability risks specified in the ILSP. A 
NDI program’s supportability objectives act as part of 
functional requirements. Supportability risks may appear in 
design by overlooking realistic operational conditions, RAM, 
Survivability, and test equipment required. NDI 
Supportability plans may not be included in Government 
requests for contractor proposals (RFPs), bidding contractors’ 
responses, and as NDI source selection criteria. ILS risks in 
NDI competitive proposals may include neglect of warranties, 
data rights, and best value procurements. 

The type and level of support affects the severity 
of impacts ILS risks have in an NDI acquisition. The user and 
PM may not clearly decide whether to use organic support, 
contractor support, or a mix of both in the ILSP as interim 
and longterm measures. Trade-offs may not be included in this 


decision. The decision should be incorporated into the 
overall acquisition strategy. Decisions on the level of 
repair affects spares and repair parts requirements. The 


sources of supply are also at issue. 
The rapid availability of NDIs might outpace their 


E\;2) 


logistics support structure in the field. An accelerated 
acquisition cycle may overpower training, maintenance, and 
logistics support activities for a new item. The NDI product 
may lack sufficient detail and realism in commercial training 
Support, training equipment and documentation. Test and 
evaluation data on design, performance, and logistics support 
may not be complete or available. Supply items may lack the 
necessary lead time to support fielding and new equipment 
training. Reduced commonality and standardization accompanied 
by increased manpower and support requirements may actually 
add costs over the product’s lifecycle (Vandeviere, p.15, 
1994). Configuration control may be complicated by rapid 
obsolescence (Cain, p.24, 1993). This then jeopardizes 
component integration and interoperability as components and 
parts are repeatedly being replaced. The supplier may cease 
production of the item which creates a problem for spares 
availability and proliferation of models. This reduces 
material readiness by creating incompatibility of system 
components. The item may require contractor maintenance 
Support in an operational environment. 

To summarize, NDI requirements, acquisition management 
environment, S&S, T&E, and ILS risk issues will challenge 
users, PMs, and potential contractors. The challenge will be 
to balance benefits in NDI acquisition costs and schedules 
with technical performance, programmatic, and supportability 
risks. Common threads tying NDI program objectives and risks 
together will be teaming among the acquisition community 
members, addressing traditional biases, market analysis, the 
Clarity and stability of user requirements, and the ability of 
the PM to cope with the accelerated features of a NDI 
acquisition strategy. 
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E. NDI ACQUISITION POLICY 


The risk management portions of DOD and U.S. Army 
acquisition policy and instruction are thorough. The NDI 
sections are treated likewise. 


1. DOD DIRECTIVE 5000.1 


DODD 5000.1 "requires the use of an existing U.S. or 
allied military or commercial system be assessed and 
thoroughly reviewed as an approach to satisfy a need or 
requirement. The directive also states that when tailoring an 
acquisition strategy to meet individual circumstances, 
consideration should be given, whenever possible and 
appropriate to maximizing practical use of commercial "off- 
the-shelf" (COTS) products" (PM Notebook, p.1.5.1-1, 1992). 


2. Government Policy 


The National Defense Authorization Act of 1987 "requires 
DOD to use NDI to fulfill needs to greatest extent possible. 
It also requires that DOD state its need in terms of functions 
ce be performed, performance required, or essential 
characteristics" (PM Notebook, p.1.5.1-1, 1992). This bill 
followed the 1986 Packard Commission’s report which proposed 
that (President’s Blue Ribbon Commission on Defense 
Management, 1986): 


DOD should make greater use of components, systems, 
and services available off-the-shelf. It should 
develop new or custom-made items only when it has 
been clearly established that those readily 
available are clearly inadequate to meet military 
requirements. 
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The commission’s recommendation to the Under Secretary of 
Defense for Acquisition was to require PMS "to receive a 
waiver before using a product made to military specifications 


if there is an available commercial counterpart". 


F. IMPLICATIONS OF RISK MANAGEMENT POLICY FOR NDI ACQUISITION 
PROGRAMS 


The paths of risk management policy and NDI policy do not 
cross. DOD policy emphasizes both risk management and NDI but 
in differing directions. Risk management policy in the 5000.- 
series keys on developmental items and acquisition strategies. 
NDI policy focuses on its risk mitigation features gained 
through use of established commercial alternatives. Cost and 
schedule benefits aside, no description or instruction for 
risk management in an NDI program appears in the 5000.-series. 
Risk, risk management, and NDI subject area searches using the 
Air Force Acquisition Model (AFAM) supplement 1.4 dated 1 
December 1993 revealed no matches between the three subjects. 

This policy void exists within the expectation that the 
PM will "tailor" the risk management approach to NDI much like 
the acquisition cycle is tailored to NDI. Tailoring is 
accepted as overlapping, combining, or omitting elements of 
the acquisition process (DODI 5000.2, p.2-6, 1994). A NDI 
acquisition strategy is a tailored version of the 
developmental acquisition process according to the type of 
program, program risks, user requirements, and needs. 
Tailoring NDI acquisitions according to program risks, 
however, iS not accompanied by risk management guidance or 
instructions in either DODD 5000.1 or DODI 5000.2. 

NDI cost, schedule, performance, supportability, and 
programmatic objectives entail certain opportunities and 


risks. Today, these risks must be managed without official 
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guidelines because DOD and the Army, though quick to embrace 
NDI benefits, have not followed their enthusiasm with updated 
policy. Dual-use technologies, NDIs, and COTS items are 
popular, in fact mandated, as the first alternative for 
filling user requirements. Policy is needed that recognizes 
the increased use of commercial technologies for military 
applications and their corresponding risks. 

The risks in NDI acquisition are not as widely described 
or documented as the benefits. The absence of documentation 
and description is due to lack of familiarity with the NDI 
acquisition risks in competitive, commercial (defense and non- 
defense related) industries by PMs, users, and DOD policy 
writers. These commercial market industries provide DOD with 
a growing number of NDI systems not initially developed to 
meet a military need. The acquisition community’s lack of 
familiarity means a lack of comfort or confidence in 
commercial practices and standards. This then creates 
mistrust and inhibits defense-commercial interaction. Without 
formal risk management policy and guideline initiatives by DOD 
acquisition participants, definitive policy and instruction 
has insufficient support or momentum to be officially 
implemented. 

In NDI programs, probabilities of adverse events and 
their consequences in requirements, the acquisition management 
environment, S&S, T&E, and ILS significantly impact the 
achievement of program objectives. PMs need more explicit 
guidance in policy and instruction regarding NDI risk 
management in the streamlined, accelerated NDI environment. 

ie: NDI Requirements. Policy does not address how 
requirements generation and formulation during the NDI Concept 
Exploration and Definition phase creates risk and how it 
should be managed. Requirements and expected capabilities are 
not compared to what is realistic and commercially available. 


Market analysis, investigation, and surveillance are not 
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adequately considered as part of the risk management process. 

2. Acquisition Management in the NDI Environment. The 
claim "this is the way it’s always done" captures the typical 
resistance inherent in the developmental perspective often 
used by PMs, users, and DTIB contractors. Risk management 
policy describes none of the interrelationships between the 
acquisition environment and its participants: PM, user, DOD, 
commercial vendors of NDIs, and Congress. Constrained 
resources, competitive pressures, extensive oversight, and 
constant change create continuous friction. These frictions 
can make it difficult for DOD’s PMs, users and commercial 
representatives to work together effectively toward evaluating 
trade-offs and satisfying user needs. The milestone reviews 
and in-progress reviews of a NDI program lack risk management 
Guidelines for examining risk management effectiveness. 
Measures of effectiveness and.measures of performance are 
absent for NDI risk management. 

a NDI Specifications and Standards. CIDs are 
unfamiliar ground to much of the acquisition workforce. 
MILSPECS and MILSTDS provide a "comfort level" to PMs telling 
the contractor specifically what to do and how to do it. 
Managing risks inherent in the use of CIDS as opposed to 
MILSPECS and MILSTDS is an unknown because extensive 
consultation with DTIB manufacturers on technical performance 
is relatively new. Existing risk management policy does not 
accommodate well the loud voice that the DTIB must have in 
establishing performance S&S for NDI items. To the contrary, 
under existing policies, DOD and its PMs have grown accustomed 
to being the dominant voice in such matters. 

4. NDI Test and Evaluation. T&E is a part of risk 
management, specifically risk assessment and analysis. Risk 
management for NDI T&E risks overlaps with requirements, the 
NDI acquisition environment, and specifications risks. DOD 


policy does not spell out guidelines for managing risks in NDI 
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testing methodology, scheduling, or integration with other 
systems in operational environments. 


5G NDI Integrated Logistics Support Planning. The 
greatest number oof risks in fielding NDI involve 
Supportability. Risk management policy neglects the market 
analysis of modification requirements, Sustainment of 


logistics and supply, and availability of the product and its 
production operations over the typical 15-20 year defense 
horizon. DOD policy overlooks risks in contractor versus 
Army - run depots, buying upgrade/pre- planned produce 
improvement (P?I) packages, spares, and technical data 
packages. The "other sources of challenges are the standard 
internal DOD processes which must be expedited or tailored to 
accommodate an NDI strategy" (Vandeviere, p.15, 1994). This 
policy gap must deal with such issues as training development, 
logistics and maintenance support, personnel management 


systems, and military force structure planning. 
G. SUMMARY 


DOD and U.S. Army risk management policy provide no 
explicit details or guidance on managing NDI-related risks. 
DOD’s shortfalls were described above. Similarly, the Army 
has not addressed the growing NDI area with risk management 
guidelines with which a PM could structure an NDI risk 
management program. With the increasing level of commercial 
products in DOD and U.S. Army inventories, it behooves them to 
initiate policy and guidelines for NDI risk management 
programs. 

The following chapter studies the efforts of the Army’s 
FAAD Sensors Program Office to manage the risks in its NDI 
acquisitions and assemble a risk management program in the 
absence of NDI risk management policy. 
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IV. THE FAAD SENSORS CASE 
A. PURPOSE 


The purpose of this chapter is to examine the risk 
Management program of an Air Defense Artillery NDI acquisition 
program office. The chapter begins with a review of the ADA’s 
critical early warning radar need upon retirement of the 
Forward Area Alerting Radar (FAAR). The review briefly 
describes air defense doctrine and early warning radar 
requirements. The chapter then introduces the Forward Area 
Air Defense (FAAD) Sensors program which includes the FAAR’s 
NDI replacements, the Ground Based Sensor (GBS) and the Light 
and Special Division Interim Sensor (LSDIS). It explores the 
risks facing the two systems. The chapter summarizes the PM’s 
overall risk management program and risk management plan. The 
chapter considers how the programs’ risks and risk management 
may have resulted in the delays in fielding early warning 
radars and the consequent ADA force readiness gap. The 
chapter concludes with a look at the support provided by DOD 
risk management policy to NDI system PMs as they structure 
their risk management programs. 


B. EARLY WARNING OVERVIEW AND FAAD ACQUISITION IMPLICATIONS 


1. The FAAR and FAAD Doctrine 


Early warning networks (EWN) support battlefield command, 
control, communications, and intelligence decisions for FAAD 
battalion commanders and Army division commanders. Early 
warning radars making up the EWN provide continuous 
Surveillance, detection, acquisition, and tracking information 
On potential air threats flying over an Army division’s area 
Of operation. Ideally, the radars operate both day and night, 
in all weather conditions, in both electronic countermeasures 
(ECM) and anti-radiation missile (ARM) environments (GBS 
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Acquisition Plan, p.8, 1990). 

Until 1990, the Army’s ADA branch relied on the AN/MPQ-54 
FAAR to provide its critical early warning information. The 
FAAR early warning system consisted of an early warning van 
with a telescoping radar pulled*by its prime mover, the Gamma 
Goat. Air intelligence preparation of the battlefield (IPB) 
and proactive air defense coverage depended on the FAAR to 
continuously broadcast early warning of approaching threat, 
unknown, and friendly aircraft. Without inherent early 
warning capabilities in ADA battalions, the battalions’ IPB 
processes and fire units’ combat effectiveness suffer. The 
result of this lower effectiveness is a higher attrition rate 
of prioritized divisional assets by enemy air threats as well 
as increased incidents of fratricide. 

The Army removed the FAAR from active service as of 31 
October 1990 (LSDIS Acquisition Plan, p. 7, 1990). The FAAR 
waS scrapped because it proved highly susceptible to ECM, 
suffered from a poor operational readiness rate, and did not 
cue ADA fire units effectively to approaching enemy aircraft 
(Stolt, June 1994). The FAAR’s Gamma Goat prime mover 
experienced consistent maneuverability limitations. The 
radar’s range limitations impaired its survivability (in the 
face of ARM). The FAAR system had incompatible communications 
configurations for digital radio transmissions and its design 
featured no air-droppable capability (Wilson, August 1994). 

No field-ready early warning replacement systems existed 
for Desert Shield and Desert Storm. Divisional ADA battalions 
deploying to the Persian Gulf either removed their FAARs from 
"mothballs" for minimal early warning capability or deployed 
without them. By the conclusion of the campaign, divisional 
ADA units would be relying on "1940's Eyeball Technology", 
commonly called binoculars, as their organic EWN capability 
(Tremmel, 1994). 
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Replacing binoculars with effective early warning radar 
technology has become an urgent requirement for the ADA branch 
and the Army (FAAD Sensors PMO, 1994). ADA FAAD doctrine 
depends on an effective early warning component to succeed. 
FAAD doctrine integrates command and control nodes, fire units 
such as Avenger teams and Bradley Stinger Fighting Vehicle, 
and early warning radars to provide comprehensive FAAD 
protection to forward deployed Army divisions. DOD and the 
Army tolerate this EWN readiness gap while replacement radars 
are tested and produced (Tremmel, 1994). The readiness gap 
left by the retirement of the FAAR mandated an effective 
defense acquisition program to fill the early warning need and 
make FAAD doctrine viable (Tremmel, 1994). 


2. Acquisition Implications 


Past FAAD acquisition program difficulties and the FAAR’s 
problems have created an environment of high risk for future 
programs and close DOD scrutiny for ADA branch PMs.  FAAD 
acquisition programs suffered a string of disappointments in 
the 1980’s and early 1990's. The Divisional Air Defense Gun 
(the Sergeant York), the Roland, the Air Defense Anti-Tank 
System, and the Fibre-Optic Guided Missile were all FAAD- 
related programs that were cancelled due to inadequate 
technical performance or high design and development costs. 

The FAAD programs’ reputation for expensive failures 
combined with the FAAR’s' lifecycle maintenance and 
Supportability woes increases the danger of performance, 
programmatic, and supportability risks. Any future ADA 
acquisition to support FAAD, whether a weapon system or radar, 
would also require close management of its cost and schedule 
risks. This environment would dictate more substantial and 


detailed FAAD Sensors program risk management. 
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A lower-risk acquisition of EWN systems requires that the 
PM recognize the Desert Storm’s-legacy. This legacy consists 
of the expectations by DOD, Army, and Congressional 
policymakers that the U.S. Air Force (as it did in the Gulf 
War) will always maintain air superiority in any conflict. 
The consequence of this assumption is that approval of major 
defense acquisition programs and resource decisions would 
neglect or discount air defense early warning as a justifiable 
need that required and deserved funding. Given past FAAD 
acquisition failures, their inadequate risk management 
efforts, and arguments against the need for Army EWN, close 
risk management is needed to prevent loss of program support 
and to control possible environmental (political) risks. 

This set of circumstances demanded that a successful EWN 
acquisition program had to field long-term, operationally 
effective, and suitable systems while effectively managing 
risk. A streamlined and tailored NDI system acquisition which 
met operational requirements and featured limited program 


risks would be ideal for the situation. 


C. THE GROUND BASED SENSOR (GBS) 


1. GBS System Overview 


The FAAD Sensors PM manages the GBS as one of two systems 
acquired to provide ADA its EWN capabilities. The AN/MPQ-64 
GBS is a three-dimensional battlefield air defense radar which 
employs a modern, phased array antenna for azimuth, elevation, 
and range data for approaching air threats. It automatically 
detects, tracks, identifies, and reports targets. It tracks 
rotary and fixed-wing aircraft, unmanned aerial vehicles 
(UAVs), and cruise missiles. It classifies targets as fixed- 
wing or rotary aircraft and differentiates air targets from 
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ground targets. It provides cueing data and track updates 
once every two seconds using digital communications to fire 
units. Its 40km range allows it to facilitate target 
acquisition beyond ordnance release lines to maximize fire 
unit reaction time and engagement ranges. Its Identification 
Friend or Foe (IFF) feature prevents fratricide by 
interrogating aircraft in terms of their coded identity. It 
tracks and searches for targets simultaneously (GBS Integrated 
Program Summary (IPS), p. G-1, 1994). 

The GBS is a NDI system. It is the AN/TPQ-36A Firefinder 
radar modified for FAAD missions and requirements (see Figure 
4). It is manufactured by Hughes Aircraft Corporation. The 
GBS features a proven system capability with extensive data 
and support information concerning Seost: schedule, 


performance, and Supportability program objectives. 
2. The GBS Acquisition Strategy 


The GBS mission need studies began in 1979 as part of an 
automation-based, ADA force modernization initiative. FAAD 
doctrine and a GBS acquisition strategy evolved from these 
studies. The acquisition strategy emphasizes systems near or 
ready for production. The requirement for a FAAR replacement 
was approved by the Vice Chief of Staff, Army on 3 September 
moo 5 . The requirement established that fielding begin in 
Fiscal Year 1991 employing an NDI sensor and common Army 
hardware and software. The Milestone III Acquisition Decision 
Memorandum approved the GBS NDI Acquisition Strategy on 19 
August 1986 (GBS IPS, p. C-1, 1994). 

The FAAD GBS Request for Proposal (RFP) was issued on 19 
April 1988. Hughes was the only respondent. Candidate 
evaluation tests proved its proposed AN/TPQ-36A radar did not 
meet requirements. The solicitation was terminated on the 


basis of unattainable performance requirements contained in 
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Figure 4 The Ground Based Sensor (Source: FAAD Sensors 
PMO, 1994) 


the ORD. Minimum operational requirements were reassessed, 
the GBS ORD was amended, and the new ORD approved by the Army 
on 29 November 1989. 

With relaxed requirements, the Army issued a competitive 
resolicitation for GBS development contract bidders which 
included a Source Selection Evaluation Test (SSET). The SSET 
consisted of proposal evaluations and field tests focusing on 
both technical and operational requirements. The proposal 
emphasized that the Government desired the "best value" GBS 
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system to satisfy its requirements (GBS IPS, p. C-2, 1994)?*. 

Seven candidates responded to the new SSET contract 
solicitation. The goal was a NDI sensor with reduced 
Government development costs and a shorter development phase. 
Hughes won the competitive evaluation and was announced as the 
contract awardee in December 1991. 

GBS development began on February 1992. Pre-production 
sensors were delivered and pre-production qualification tests 
(PQTs) were successfully completed as part of the program’s 
Phase I. The initial sensors met performance and operational 
requirements in accordance with exit criteria. Phase II 
testing, which concerns the integration of GBS with FAAD 
command, control and intelligence (FAADC?T) systems, Avenger, 
and Man Portable Air Defense Stinger (MANPADS) teams will 
continue until pre-production sensors are completed and 
delivered in February 1996. GBS fielding will finally begin 
met 997 (GBS IPS, p. C-4, 1994). 


D. THE LIGHT AND SPECIAL DIVISION INTERIM SENSOR (LSDIS) 


1. LSDIS System Overview 


The FAAD Sensors PM manages the LSDIS as the second of 
the two systems being acquired for FAAD EWN capabilities. 
Whereas the GBS is initially designated for ADA units 


Supporting armored, mechanized, and armored cavalry divisions, 


* Note: Best value is defined as "acquiring a production 
ready system through full and open competition that best meets 
the Army’s needs considering cost, schedule, performance and 
risks" (LSDIS Acquisition Plan (AP), p. 8, 1990). The best 
value approach strives to meet Army needs by the product 
itself or with a P3I program. It differentiates minimum 
requirements from objective requirements. The contractor can 
meet minimum requirements rather than objective ones. The 
disadvantage is that the Government is not assured of ever 
reaching those objective requirements. 


Syl 


the LSDIS is dedicated primarily to ADA battalions in light 
infantry, airborne, and air assault divisions and contingency 
corps. The LSDIS is a two-dimensional air defense radar that 
alerts fire units with azimuth and range data on approaching 
short-range, low-altitude aircraft. It is a highly mobile, 
lightweight, manportable system which detects and 
automatically classifies fixed-wing and rotary aircraft up to 
a range of 20km (see Figure 5). It is an interim system until 
a light-weight version of the GBS is fielded (no earlier than 
Fiscal Year 00) (LSDIS Acquisition Plan, p. 7, 1990). 

The LSDIS is a NDI system. It is intended to support 
rapid-deployment light and special divisions in low-intensity 
conflicts with early warning alerts of incoming threat 
aircraft. It is classified as an urgent, directed procurement 
to expedite its fielding (LEWDD Directed Procurement 
Memorandum, p. 1, 1990). 


2. The LSDIS Acquisition Strategy 


The LSDIS NDI acquisition strategy consists of a 
competitive procurement reflecting the FAAR’s retirement, 
pursuit of the best technology available, and minimizing 
expected cost and schedule. The directed procurement "label" 
originated from the urgent need to field an early warning 
sensor to light and special ‘division ADA units until a 
lightweight version of GBS could be deployed. 

The LSDIS acquisition cycle began with the approval of 
the Lightweight Early Warning Sensors Required Operational 
Capability (the LSDIS’s original ORD) on 22 June 1990 and the 
Directed Procurement of the Lightweight Early Warning 
Detection Device (LEWDD) Memorandum published 26 July 1990. 
These were based on the technical requirements of the U.S. 
Marine Corps LEWDD program. The directed nature of the LSDIS 
procurement emphasized an accelerated solicitation and source 
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Figure 5 The Light and Special Division Interim Sensor 


selection process (LSDIS IPS, p.4, 1993). 

The FAAD Sensors PM recognized that not all system 
requirements would be met. This was particularly true with 
objective requirements. Proposal evaluations were based on 
best value and considered tradeoffs between objective 
performance requirements and current technical capabilities 
(minimum requirements), cost, and schedule (LSDIS AP, p.10, 
ooo) . 

The LSDIS RFP was released on 30 July 1990. Lockheed 
Sanders and Lear Siglar were the only bidders. The LSDIS 
program emphasized a streamlined NDI approach that allowed the 
Government to rapidly meet its urgent need. The competition 
emphasized that bidders provide reduced cost, efficient, and 
"best value" systems. Lockheed Sanders was awarded the best 


value contract on 13 November 1990. 
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LSDIS produztt testing began after the contract award in 
May-June 1991. The LSDIS contract required a First Article 
Test (FAT) and a PQT after which additional production 
contract awards would be issued. The date of the first unit 
(to be) equipped (FUE) was late 1992 with fielding to be 
complete in 1994 (LSDIS AP, p.7, 1990). The LSDIS FAT was 
successfully completed during 1 July 1991-15 November 1992 
with the PM conditionally accepting the first unit. The PQT 
occurred from 2 March-7 May 1992. The LSDIS failed to meet 
operational requirements and system specifications due to 
false target rates, reliability-availability-maintainability 
(RAM) shortfalls, and azimuth alignment problems. A second 
PQT initiated on 7 June 1993 revealed all shortfalls were 
corrected and no further operational failures were exposed. 
The contractor’s inability to meet operational requirements 
resulted in higher production costs and schedule delays of one 
year (LSDIS IPS, p.10;7e19923 5 

The LSDIS NDI acquisition strategy required that 
production begin only after successful PQT completion. The 
LSDIS directed acquisition authorized limiting specification 
Standards, test plans, and contract data requirements in its 
RFP as a means of saving costs. Waivers of those items 
designated as non-cost effective contract requirements were 
intended to shorten the program’s duration. The acquisition 
strategy stressed "Streamlined" program planning, testing, and 
ILS (LSDiIs LPS, pet07 i3os)— 


E. RISKS AND RISK PERCEPTIONS IN THE FAADS SENSORS PROGRAMS 


1. GBS Acquisition Program 
a. Technical Risks 


The revised requirements published in the GBS 
resolicitation reflected the Government’s best value approach. 


54 





New minimum operational requirements allowed the contractor to 
make tradeoffs in technical and performance capabilities. The 
GBS design was simplified as a two- (azimuth and range only) 
rather than three-dimensional radar to meet cost and schedule 
objective requirements (Eison, 1994). Warranties for 
reliability and performance of accepted units assured 
compliance with all minimum requirements. The PM rated GBS 
technical risk as low to moderate for meeting minimum 
requirements (GBS Acquisition Plan, p.9, 1990). 

The PM rated the risk of meeting technical performance 
objective requirements broadly as being low to high. This 
rating encompassed the known, current technical performance of 
the radar (based on minimum requirements) while accounting for 
unknowns in future, P*I-based radar performance (objective 
requirements) . 


b. Cost Risks 


The PM rated the GBS cost risk as low to moderate. 
The GBS design was based on the Firefinder configuration which 
initially reduces the development requirements and costs. 
Schedule and performance risks are also dampened by the mature 
design. The cost risk rating considers the range of possible 
performance capabilities and contracting costs resulting from 
a best value approach. Cost history and production experience 
with the AN/TPQ-36A Firefinder indicate low cost risks in 
production. The GBS PM used a Firm-Fixed-Price (FFP) contract 
to contain Government cost risks. The PM’s conclusion was 
that a NDI acquisition strategy would procure the best 
technology available. A reasonable contract price was 
expected through the competitive bidding process (GBS AP, p. 
mo, 1990). 

The PM raised the GBS cost risk to moderate when the 
p?t program was added. These cost risks reflected research 
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and developmen: (R&D) and design modifications required to 
achieve objective performance requirements. They also 
consider a different contract type. Hughes Aircraft 
Corporation would expect a Cost-Plus contract to cover its R&D 
costs and higher risks assumed in attaining higher objective 
performance requirements. If Hughes or a competitor agreed to 
a FFP contract (given that some P?I developments have existing 
technological solutions), cost risks would be lower for the 
Government and moderate (rather than high) for the contractor. 
GBS’s objective performance requirements would be pursued 
early in the acquisition cycle if additional funding were 
approved for the GBS P?I program (GBS AP, p. 11, 1990). 


c. Schedule Risks 


The PM rated the GBS schedule risk as low for 
meeting a two year delivery schedule for a first production 
sensor upon contract award. Market surveys supported this 
assessment. Surveys revealed the risks of late Fielding due 
to completion of a P°I program increased schedule risks to a 
range of moderate to high. The PM intended to pursue mature 
technology as part of the P?I. Schedule risks involving the 
completion of the longer-range Boi program depended on the 
increased technological complexity of the sensor and expected 
costs to achieve those objective performance requirements (GBS 
RD. 1 1990) = 


ad. Supportability Risks 


The PM’s management of supportability risks is based 
on acquiring a validated technical data package (TDP). The 
purchase of a TDP is intended to reduce risks concerning 
future repair parts by Government ownership of technical 
documentation. Future repair parts procurements could then be 
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competed to find adequate support contractors to provide 
uninterrupted parts over the systems entire lifecycle (GBS AP, 
med, 1990). No risk rating was assigned to system 
supportability risks in the GBS Acquisition Plan. See Figure 
6 for the GBS Risk Assessments. 


2. LSDIS Acquisition Program 
a. Technical Risks 


The PM rated the LSDIS technical risk as low to 
moderate. As with the GBS, the LSDIS program followed a best 
value acquisition approach. This approach allowed the PM, 
armed with market survey results, to compete the procurement. 
However, LSDIS performance would be tested during sensor 
production. The contractor would warranty reliability and the 
fielded systems’ performance after acceptance. Risk 
assessments would be based on FAT and POT results (LSDIS AP, 
Meet, 1990). 


b. Cost Risks 


The PM rated cost risk as low based on the LSDIS FFP 
contract and competitive procurement strategy. LSDIS system 
configurations represent low cost risk with existing cost 
history and production background already established. Price 
again was a factor in the best value approach to reduce the 
Government’s cost risks. Price considerations under a 
competitive NDI acquisition strategy was not anticipated to 
raise cost risk above a low rating (LSDIS AP, p.11, 1990). 
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Figure 6 The GBS Risk Assessments (Source: GBS IPS, 1994) 
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c. Schedule Risks 


The PM rated schedule risk as being moderate based 
on the urgent and accelerated schedule. The contractor was 
expected to meet LSDIS’s directed procurement fielding dates 
one year after contract award (LSDIS AP, p.11, 1990). 


d. Supportability Risks 


No description of LSDIS supportability risks or risk 
reduction measures is in the LSDIS AP. Support is rated as 
moderate in the LSDIS IPS Risk Assessment. The 1990 LSDIS 
Light Sensor Market Study did not address contractor logistics 
Support. See Figure 7 for the LSDIS Risk Assessments. 


F. THE FAAD SENSORS RISK MANAGEMENT PROGRAM 


1. FAAD Risk Management and DOD Policies 


The FAAD Sensors PM believes NDI acquisition strategies 
equate to risk mitigation (FAAD Sensors PMO, August 1994). 
Both the GBS and LSDIS risk management programs demonstrate 
the characteristics described by DODD 5000.1 and DODI 5000.2. 
Both the GBS and LSDIS risk management programs outlined 
performance, cost, and schedule risks. Supportability risks 
are mentioned in passing. The FAAD Sensors PM used a 
streamlined risk identification, assessment, and reduction 
process as a risk management program. The GBS and LSDIS risk 
identifications and ratings and their rationale are simple and 
aarect. 

DODI 5000.2 does not explicitly outline a NDI risk 
management program for NDI PMs. Army Regulation (AR) 70-1 
makes no explicit connection between NDI and risk management. 
No quantified or qualified risk analysis accompany the GBS and 
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Risk Assessment 


Light and Special Division Interim Sensor (LSDIS) 


1. Risk Assessment 
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technology used 
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Figure 7 The LSDIS Risk Assessments (Source: LSDIS IPS, 1994) 
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LSDIS risk assessments to explain how they were obtained. 
Consideration of NDI-specific risks in requirements, 
specifications, T&E, and ILS are not evident. This is because 
existing policy does not describe how specific a streamlined 
risk management approach must be, let alone refer to any NDI- 
specific risks. DOD policy provides no NDI risk management 
references for NDI PMs to cite. Due to this lack of guidance, 
NDI solutions to urgent needs and force readiness gaps can 
lead to incomplete risk management programs. PMs may tailor 
developmental item risk management guidelines and risk 
assessments formats found in DODI 5000.2 and DOD 5000.2-M to 
Suit NDI program needs. However, tailoring developmental risk 
Management policies does not assure successful NDI risk 
management. 

The NDI GBS and LSDIS programs constitute tailored, 
streamlined acquisitions of mature technology according to DOD 
and Army streamlining guidance. The FAAD Sensors PM’s risk 
Management practices incorporate tailoring and streamlining in 
his simplified identify-assess-reduce structure. The GBS 
acquisition strategy indicates that "the Army prefers to buy 
systems already designed, developed, tested, ange a0 
production, or at least where principal components are in 
production" (GBS IPS, p.C-4, 1994). Any risks involved with 
such an approach are not covered in an Army policy or 
memorandum, despite NDI’s popularity in Army acquisitions. 

DOD 5000.2-M does not outline a NDI risk assessment 
format. The GBS and LSDIS risk assessment formats are broad 
but follow the developmental risk assessment structure found 
mi DOD 5000.2-M. Any application of DOD 4245.7-M risk 
identification and assessment procedures to the indicated GBS 
and LSDIS cost, schedule, performance, and supportability 
objectives are tailored or overlooked. The DSMC Risk 
Management Guide and the AFAM similarly do not cover NDI risk 


Management policy, plans, or risk assessment formats. These 


ou 


possible risk management references are not mentioned in the 
GBS or LSDIS risk management documentation. 


2. The GES Risk Management Program 


The GBS risk management program’s risk descriptions in 
its acquisition plan and risk assessments in its IPS meet 
minimum DOD policy requirements. The GBS risk management 
program uses the DOD 5000.2-M risk assessment format. The 
GBS’s risk management program meets policy requirements by 
articulating risk planning, assessment, and reduction 
measures. 

The PM implemented technical risk reduction by using a 
two-tiered approach to technical performance. The two tiers 
are minimum and objective technical performance. A system had 
to perform within this range to meet the Army’s minimum 
requirements. The GBS contract bidders would then be 
considered as qualified potential candidates. The PM’s market 
surveys indicated low risk of meeting the minimums. Higher 
risk requirements were designated as objective or "desired" 
requirements rather than setting them as minimums as in the 
original RFP. Technical risk was reduced using testing during 
source selection. Production contract options would not be 
exercised unless pre-production systems completed technical 
tests successfully. These measures reduced risks in fielding 
production units which did not comply with warranties on 
required performance levels (GBS AP, p.10, 1990). 

To meet objective or "desired" requirements meant the PM 
had received funding approved for a GBS p*T. The GBS P?I 
program is currently unfunded. This programmatic uncertainty 
accounts for the GBS configuration changes to the HMMWV prime 
mover (low risk), radar hardware and software performance 
modifications (moderate risk), and future-based radar 


modifications to support the GBS’s projected role as a "common 


62 





@erps sensor" (GBS AP, p. 9, 1990). 

The risk of publishing unattainable requirements in the 
Original solicitation created follow-on risks to GBS 
performance, cost, schedule, and programmatic objectives. The 
sole bidder to the original RFP, Hughes, did not meet the user 
requirements during candidate system testing. Effective 
market analysis may have revealed that the combination of 
multiple systems’ characteristics stated in the ORD did not 
exist in any one radar configuration. A complete market 
analysis could have indicated problems with the original 
requirements in 1985. 

The new ORD and resolicitation of the GBS contract began 
four years later. The competition for contract award was not 
completed until 1991. Six years in schedule were added. The 
program management costs now included the resolicitation and 
re-evaluation of candidate radars based on a new ORD. The new 
schedule extended far beyond the retirement date of the FAAR 
and a FAAD EWN capabilities gap developed. The GBS’s 
objective performance requirements now hinge on unfunded 
product improvements. The impact is seen in the PM being 
forced to trade off objective performance in order to expedite 
fielding a radar with the minimum required performance 
Capabilities. The programmatic risks were realized when DOD 
decreased production funding and the number of systems it 
would buy from 77 to 57 in 1994 (Department of the Army ADA 
Highlights Memorandum, p. 6, June 1994). 

The GBS NDI acquisition strategy emphasizes the benefits 
of NDI streamlining described earlier. Although minimum 
performance requirements were met during SSET, the GBS best 
value procurement did not aoe all objective performance 
requirements. The PM and user accepted technical performance 
trade-offs. These decisions concentrated on performance that 
maximized mission support of FAADC?I and cueing FAAD weapon 
systems rather than maximizing GBS radar performance. 
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Deferred performance objectives may cause delays in procuring 
GBS P*I longlead items and eventually result in a production 
stoppage. 

The problem with a production stoppage is that resulting 
production line layoffs, retraining, facility overhead costs, 
and start-up costs risk further schedule delays and cost 
escalation to both the prime and subcontractors (GBS IPS, p. 
C-7, 1994). The traditionally low-risk NDI cost and schedule 
features have also contributed to greater technical 
performance and programmatic risks for the GBS program. The 
PM was forced to "broad brush" the technical performance risk 
rating based on the uncertainty created by an unfunded P?I 
program. 

The FAAD Sensors PM currently waits for additional 
funding approval to incorporate product improvements into the 
GBS configuration. Without the improvements, the ADA branch 
fields a minimum-capabilities system, delays procurement of 
long-lead items, and faces possible production shutdowns. 
With the improvements, the GBS schedule grows again, incurs 
more costs spread over fewer systems, and ADA endures more 
pressure to sell the program and justify its need. 

The PM planned to reduce GBS cost risks based on the type 
of contract awarded for both production and PI. Pre- 
production and production contracts were FFP with performance 
warranties. The contractor assumed the cost risks. The 
Government could reject options that were too risky. A Cost- 
Plus Incentive Fee award would be based on levels of proposed 
performance compared to desired requirements (GBS AP, p.11, 
1990). Cost containment decisions in not exercising product 
improvement options and the use of a FFP contract limited 
Government risk and enhanced program support in the short 
term. 

The PM’s schedule risk reduction effort relied on the 
best value NDI acquisition strategy. Potential contractors’ 
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proposals considered production and delivery schedules. GBS 
schedule risks were decreased by restricting production starts 
until the completion of the pre-production program. This risk 
reduction measure had its own drawback, however, of extending 
the program’s length over an unacceptable period of time (GBS 
AP, p.11, 1990). 

Supportability risks were assessed with limited depth. 
The Firefinder radar has a previously established support 
structure, supply and parts channels, and a training support 
package. While these could be tailored to the GBS fielding, 
support, parts flow, and training programs must be oriented to 
FAAD missions and roles. 

Risk minimization by the FAAD Sensors Program Office 
considered other components besides objectives. The best 
value competitive acquisition strategy incorporated proposed 
contractor risk reduction measures into the source selection 
evaluations and contract proposal evaluations. The PM’s test 
program includes a logistics demonstration in conjunction with 
operational testing to verify ILS capabilities and support 
concepts. 

3. The LSDIS Risk Management Program 


The LSDIS acquisition program’s risk descriptions in its 
acquisition plan and risk assessments in its IPS meet minimum 
DOD policy requirements. The LSDIS risk management program 
generally follows the DOD 5000.2-M risk assessment format. 
The waiver of non-statutory requirements (due to its urgent 
and directed procurement classification) reflects the more 
Simplified and less detailed risk assessment structure. 

The PM employed a LSDIS technical risk reduction effort 
Similar to the GBS. program. The Army’s' performance 
requirements spanned the range of essential requirements and 
desired capabilities. Minimum performance requirements 
consisted of capabilities and characteristics needed by the 
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user. Capabilities described in contractor proposals that 
exceeded the basic requirements were considered objective 
performance requirements. The LSDIS contract awardee would be 
selected based on the best value of a proven NDI capability. 
This technical risk minimization complemented the production 
readiness assessment during which the bidders would "prove 
out" their production units in actual demonstrations. The 
production assessment would be reinforced by the FAT and PQT 
(LSDIS Acquisition Plan, p.11, 1990). 

Risk reduction in the LSDIS acquisition concentrated on 
technical and cost risk. Schedule risks, although classified 
as moderate, were accepted based on the urgent, directed 
procurement. The limited quantity and urgent procurement 
classification allowed maximum use of waivers for all 
nonstatutory requirements (LSDIS Acquisition Plan, p.12, 
199 0)e 

The LSDIS requirements rjisk evolves from the use of 
Marine Corps requirements. The LEWDD Memorandum which served 
as the basis for the LSDIS requirement was followed by a short 
market analysis but, like the original GBS solicitation, drew 
only one bidder, Lear Siglar. With competition and best value 
common themes in NDI acquisitions, the contract was amended to 
include a competitive procurement and interoperability with 
FAADC?I at the direction of the Army Acquisition Executive 
(LSDIS Quick Report, 1993). These changes delayed the 
solicitation, added competition factors (which Lockheed 
Sanders then won, largely on the merit of its proposal (Eison, 
1994)), and entailed greater market analysis of contractor 
testing and support capabilities. 

Test and evaluation and specifications and standards 
risks surface with the emphasis on an urgent LSDIS NDI 
procurement. The directed procurement streamlined and waived 
certain aspects of the program and its documentation. The 
accelerated LSDIS acquisition program conducted performance 
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and design testing after contract award rather than during 
candidate evaluations. This did not allow for verification of 
System performance, production capabilities, or support 
structures until after manufacturing began. Once testing did 
begin, the contractor’s design did not meet specifications 
@ering POT. Operational requirements were not met. 
Integration and interoperability with FAADC?I were also 
overlooked in the requirements: Integration with the other 
components of FAAD battalions emerges as a performance 
specification issue as well. Market analysis with greater 
depth may have preempted these shortfalls. 

The LSDIS NDI acquisition management environment entailed 
greater risks than anticipated. Test and evaluation, 
specifications and standards, ILS, requirements, and NDI 
acquisition planning were streamlined to meet cost and 
schedule objectives. LSDIS technical performance has 
suffered. Despite the streamlining steps, which were well 
known to all LSDIS acquisition participants, the contractor’s 
product difficulties caused schedule slippage and cost 
increases. The short term gains will be justified only if the 
amended portions of the LSDIS acquisition cycle do not return 
to haunt the PM. The LSDIS remains fully funded for 40 
sensors (Department of the Army’ADA Highlights Newsletter, p. 
April, 1994) 

The urgent nature of this NDI procurement, similar to the 
GBS, required more strict risk management. The accelerated 
NDI acquisition strategy jeopardizes the ease with which 
program objectives are expected to be accomplished. Test and 
evaluation and integrated logistics support plans may be 
rushed to completion for the sake of cost and schedule 
objectives. The verification of performance and support 
requirements in accordance with pre-determined specifications 
and standards may become paper drills rather than evaluations. 
Market analysis may also be caught in this compressed 
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acquisition cycle, and critical performance, production, ILS, 


and program issues may be overlooked. 


4. The FAAD Sensors Risk Management Plans 


The FAAD Sensors PM did not publish risk management plans 
for either GBS or LSDIS. A risk management program does not 
imply the existence of a explicit risk management plan (RMP) 
to outline the program. Risk planning, as required by DODD 
5000.1 and DODI 5000.2, does not require a PM to release an 
RMP. The decision is left to the PM. Tailoring a Wibi 
acquisition strategy to meet program requirements allows the 
PM to forego a published RMP. Tailoring, however, leaves a 
program vulnerable to NDI-specific risks in requirements, 
ILSP, T&E, S&S, and the NDI acquisition management 
environment. These contribute to risks in its cost, schedule, 
performance, and supportability objectives. 

The absence of DOD or Army NDI risk management policy or 
outlines leaves the burden on the PM. The DSMC’s proposed RMP 
outlined in its Risk Management Concepts and Guidance 
publication assumes a developmental program application. It 
neglects the risks and risk management needs unique to the NDI 
acquisition program. The AFAM likewise overlooks the needs of 
NDI PMs implementing risk management policy. 

Assistance to NDI system PMs in the form of NDI risk 
management templates, sample NDI RMPs, and specific DOD and 
Army NDI risk management policy would facilitate NDI risk 
management practice. These measures would also counteract 
PMs’ inclinations to treat risk management in their NDI 
programs aS a costly, time-consuming, non-value added effort. 
The desire to tailor and streamline developmental risk 
management policy rather than work with explicit NDI risk 
Management guidance leaves the PM free game to auditors, 
evaluators, and stakeholders in a NDI acquisition environment 
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that may grow less forgiving. The NDI templates and RMPs 
would serve as readily available tools to quickly and 
effectively assemble productive risk management programs that 
successfully grasp and control NDI risks. Improved DOD and 
Army NDI risk management policy would reinforce their 
legitimacy. 

The ADA’s FAAD doctrine depends on EWN. The FAAR 
retirement generated a credible mission need. The need was 
approved by the Army. Requirements were specified to meet the 
urgent needs. The Gulf War highlighted the pursuit of timely 
system fielding (programmatic risk). However, the original 
GBS requirements were not well written and reflected 
insufficient market analysis into available early warning 
radar technology (requirements and market analysis risks). 

The GBS and LSDIS NDI early warning radars provided 
initially attractive solutions to ADA’s FAAD and force 
readiness gaps. Early operational assessments, SSETs, FATs, 
and POTs, however, uncovered weaknesses in both sensor designs 
that confirmed best value for the Government but did not yield 
desired performance (T&E and technical risk). The accelerated 
acquisitions had rushed the "limited" research and development 
process to meet urgent fielding requirements at the expense of 
confirmed, commercially-validated requirements and performance 
risk "multipliers" in T&E, S&S, and ILS. 


G. SUMMARY 


The GBS and LSDIS risks are described by the PM in terms 
ona cost, schedule, performance, and Supportability. 
Assessments and analysis of these risks are restricted to 
outlines since no RMP is used for either program. This is 
understandable in light of the DOD and Army policy voids and 
NDI tailoring flexibilities. The problem is that a "pick and 
Choose" policy environment frustrates a PM’s definitive 
efforts to build an effective risk management program and to 
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write an explicit RMP. The pressure to buy NDI and ramrod 
commercial specifications and standards into contracts will 
only result in more tailoring and streamlining of RMPs. 
Urgent and best value labels put on acquisition programs will 
continue the RMP and risk management tailoring trend in NDI 
acquisitions. The cost and time required to staff RMPs will 
only be justified when better tools, policies, and incentives 
exist to prepare detailed risk management programs. 
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V. MERGING THE TWO PATHS INTO ONE 
A. PURPOSE 


The purpose of this chapter is to analyze DOD risk 
management policy as it stands and to propose changes that 
Support NDI acquisition programs. The FAAD Sensors risk 
management lessons learned appear first in this chapter. The 
importance of these lessons is reinforced by examining the 
risk management policy and the lack of detail it provides 
concerning NDI. The chapter introduces a dendritic diagram 
that breaks down developmental and NDI risk management 
requirements. It then considers improvements to portions of 
DOD risk management policy concerning NDI risks. NDI risk 
management tools are offered that can help PMs in managing 
risks in their NDI acquisition programs. It includes the 
potential benefits in NDI risk management templates, risk 
management plans (RMPs), and‘watch lists. It looks at 
possible revisions to the steps in the NDI acquisition cycle. 
The chapter concludes with a summary of where policy and 
procedure changes will expedite and increase the value of NDI 


risk management programs and plans. 
B. FAAD SENSORS LESSONS LEARNED IN RISK MANAGEMENT 


Several lessons can be gained from the FAAD Sensors 
program in the realm of risk management. The FAAD Sensors PM 
manages the GBS and LSDIS NDI programs using DOD’s 
developmental program-based risk management policies. The 
FAAD Sensors program risks are categorized by cost, schedule, 
and performance objectives. Supportability risks are briefly 
mentioned. Programmatic risks are not specified. The PM uses 
DODD 5000.1, DODI 5000.2, and DOD 5000.2-M risk management 
guidance in managing his NDI programs. 

In both the GBS and LSDIS programs, the Acquisition Plans 
(APs) identify risks, rate them, and describe risk handling 
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and reduction. Risk assessments are furnished in the 
programs’ Integrated Program Summaries (IPS) according to DOD 
5000.2-M formats. The NDI acquisition strategy in itself is 
viewed by the PM as risk mitigation. With these risk 
management program characteristics established, the following 
lessons emerge: 

1. The FAAD Sensors PM relied on developmental program- 
based risk management policy direction and instruction because 
no explicit DOD NDI risk descriptions, risk management 
policies, or risk management tools such as NDI risk templates 
exist. 

2. The PM tailored and streamlined DOD risk management 
policy, risk assessment formats, and risk management tools to 
meet the demands of his NDI programs’ needs. While executing 
his risk management duties prescribed by DOD, the PM is 
vulnerable to second-guessing by auditors and other critics 
because specific NDI policy references that could be useful to 
justify risk management decisions and actions do not exist. 

Tailored risk management practices and streamlined risk 
management documentation are acceptable in NDI DOD acquisition 
management. This is true provided the PM’s risk management 
process does not break down. In an era of constrained 
resources, smaller RD&A budgets, and significant pressures to 
contain risks and buy NDIs, unsuccessful NDI risk management 
programs spell program failure. More explicit NDI risk 
management policy would protect programs if it required 
structured risk management planning in the form of an RMP. 
PMs could also protect the validity of their risk management 
efforts by using and referencing templates, program-specific 
watch lists, risk models, or some combination of these tools. 
Thus, the PM could have a safety net while walking the NDI 
risk management high wire, with references and justification 


for risk ratings, assessments, and handling measures. 
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2S. The GBS and LSDIS IPS risk assessments rate the 
majority of risks low while their APs describe risks in ranges 
(generally low to moderate). The lack of consistency might be 
related to the difference in publication dates of the 
documents that guided these assessments. GBS and LSDIS APs 
published in 1990; the GBS IPS in 1994; the LSDIS IPS in 1993. 
In NDI program management such as with the FAAD Sensors case, 
the PM introduces programatic risk by classifying more thana 
few program risk areas as moderate or high. NDI hold the 
reputation of being low risk, and to rate risks otherwise may 
draw audits, criticism, more documentation and meetings, and 
unwanted stakeholder involvement. Uniformity of program risk 
reviews prepared by contracting officers in APs that then 
translates directly to the PM’s risk assessments in IPSs 
reduces the need for such scrutiny. 

This uniform treatment of program risks and the plans to 
manage them could be most appropriately gained through a 
published RMP which serves as a focal point for all risk 
management issues and actions. A published RMP can be easily 
referenced by IPSs and APs. The PM’s RMP could then bind AP 
and IPS risk management material together. This 
decisionmaking support tool could remain as an available 
program management document from one milestone decision review 
to the next. Its proper use would require training and 
education in NDI risk management planning. Examples (see 
Figure 8 and 9) are shown on the accompanying pages. 

4. The FAAD Sensors PM does not consider NDI sources of 
risk that could potentially wipleoueoucceoseul achievement of 
cost, schedule, performance and supportability objectives. 
Requirements, NDI Acquisition Management Environment, Test and 
Evaluation, Specifications and Standards, and Integrated 
Logistics Support risks acute in NDI acquisitions are not 
recognized or anticipated in any GBS or LSDIS IPS or AP. This 
is because such risk areas are only now being identified. 
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Although tnere may be evidence of these risks in the two 


systems’ test and evaluation master plans, systems engineering 
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Figure 9 DSMC’s Systems Engineering RMP Format 
(Source: DSMC Systems Engineering Management 
Guide, Chapter 15, 1990) 


master plan, and integrated logistics support plans, these 
considerations are not cross-walked with AP and IPS risk 
documentation. 

These lessons are not cited at the FAAD Sensors PM’s 
expense. They are learned with his help. Previous FAAD 
programs such as the Sergeant York, the Roland, and Air 
Defense Anti-Tank System were doomed by inadequate risk 
management programs whereas the GBS and LSDIS programs are 
consistent with what DOD risk management guidance requires in 
aint . The FAAD Sensors NDI acquisition approach is 
consistent with the DOD SD-2 Buying NDI Guide which specifies 
market analysis, requirements development, solicitation, 
source selection, product assurance (specifications), T&E, and 
ILS procedures (Office of the Secretary of Defense for 
BmeeGuction and Logistics, 1990). Any trade-offs made in 
staffing a tailored, streamlined NDI risk management program 
versus other program requirements is not the focus of this 
thesis. However, the FAAD Sensors case, like the SD-2 and 
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other DSMC and DOD publications’,, does communicate the impact 
of NDI sources of risk and their consequences in terms of 
fielding delays, performance shortfalls, cost increases, 
Supportability concerns, and programmatic challenges (e.g. 
funding shortfalls that reduce the number of systems to be 
fielded, unfunded P?I, the push for CIDs over MILSPECS and 
MILSTDS). The ADA user community has as much impact on the 
control of these risk events as the PM. The PM’s user has an 
obligation to effectively research and validate system 
requirements that directly affect its divisional tactical 
doctrine, force readiness, and potential combat capability 


gaps. 


C. THE DIVERGING PATHS 


The crux of the NDI risk management issue is that no risk 
management policy explicitly addresses NDI acquisitions. Risk 
management and NDI policy do not converge. DODD 5000.1 and 
DODI 5000.2 invite tailoring of their risk management policies 
to fit the needs of the individual programs. The implicit and 
somewhat erroneous assumption in these policies is that risk 
management is practiced for developmental programs with 
complete acquisition cycles. NDI programs do not correspond 
to these assumptions. DOD policies neglect NDI risks and risk 
management practices. 

NDI acquisition riskS are unique. They reflect 
accelerated, sometimes urgent procurements with abbreviated 
acquisition cycles to meet user needs and requirements. NDI 
acquisitions tailor and streamline acquisition policy and 
particularly, risk management policy, and make tradeoffs among 
program objectives. While NDI programs do mitigate risks in 
cost and schedule, their technical performance, 
Supportability, and programmatic risks are heightened. The 
consequences of risk events in these areas can have tremendous 
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affect on cost and schedule in the long run. DOD’s challenge 
is to bridge the gap between developmental acquisition risk 
Management policy and practices and those that apply 
specifically to NDI risks. Effective application of NDI-based 
risk management policy and practice can save PMs valuable 
time, money, and manpower resources while assisting them in 
effectively meeting program goals and objectives. 

Risk management programs and NDI solutions are popular 
acquisition issues whose paths must cross to serve the best 
interests of NDI PMs, their programs, and their customers. At 
present, however, the two paths lack any formal policy 
connection. The point is not to invite more regulation, but 
instead to appeal for official documentation and guidance that 
corresponds with today’s acquisition environment and its 
emphasis on acquiring state-of-the-art technologies today as 
sources of tomorrow’s Army systems. By updating and 
correcting the oversights in NDI risk management policy and 
guidelines, PMs can more quickly and accurately assemble risk 
management programs and concentrate on program objectives with 


an appropriate NDI risk management baseline. 


D. A DENDRITIC APPROACH 


DOD risk management policy exists in DODD 5000.1, page 1- 
4, and DODI 5000.2, Part 5B. The Army published its 
acquisition policy with references to risk management in Army 
Regulation (AR) 70-1. Its Department of the Army Pamphlet (DA 
PAM) 70-2 on Risk Management is due in February 1995. Army 
acquisition risk management policy implicitly corresponds to 
fae mirrors that of DOD. Army risk management policy 
pertaining to NDI in AR 70-1 is not articulated despite the 
popularity of NDI acquisitions and Army NDI initiatives. DOD 
risk management policy according to DODD 5000.1 and DODI 
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5000.2 relies officially on risk management tools such as 
templates outlined in DOD 4245.7-M and the AFAM and 
unofficially with DSMC’s risk management plans, risk matrices, 
and watchlists found in its risk management guide. Army PMs 
likely will tailor DOD guidelines for implementing risk 
management to suit individual program needs. 

A model such as a dendritic diagram can lay out the DOD 
risk management issue, itS scope, criteria, measures of 
effectiveness and performance, and the rationale for NDI PMs 
to have more explicit risk management policy. PMs could then 
refer to NDI risk templates and other tools to facilitate 
effective risk management. 

Operational and design requirements, NDI acquisition 
management, specifications and standards (for design and 
manufacturing of standard and non-standard parts), test and 
evaluation (including FATs, PATs, and PQTs) and ILS all 
represent Significant NDI risk areas lacking templating in the 
DOD 4245.7-M and articulation in DSMC risk management plans. 
A dendritic layout of risk management policies and the need 
for NDI risk management guidance could be outlined as seen on 
the following page in Figure 10. 


E. DOD POLICY REVISIONS 


NDI PMs currently conduct risk identificaumenmy 
assessment, handling, and documentation according to DODD 
5000.1 “and DODI Sc0ge2- In doing so, the PM applies 
developmental acquisition risk management policy and 
procedures focusing on traditionally developmental program 
risks to nondevelopmental systems. The point is that, as seen 
with the FAAD Sensors PM, many NDI acquisition PM’s are forced 
to "think on the move" without explicit directions or road 
maps. Lacking DOD or Army NDI risk management policy, 
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instructions, templates, or even unofficial emphasis in DSMC 
risk management publications, PMs will continue to rely on gut 
feelings, developmental lessons learned, and incomplete 
references. The approach of tailoring and streamlining 
developmental risk management policy and practice to suit NDI 
program requirements puts NDI PMs and their acquisition 
programs on a high wire without a net. Program risk 
management can easily be second guessed because no official 
references can be cited to justify risk management decisions. 


Lack of published NDI RMPs increases this risk exposure. 


1. DODD 5000.1 Risk Management Policy 


DOD’s Acquisition Policy Directive 5000.1, Pane 
contains no explicit guidance directed to NDI risks and NDI 
risk management. The implicit developmental (as opposed to 
NDI) tone in the risk management policy has not kept up with 
the push to satisfy requirements using mature technology 
acquisitions from Government agencies and/or commercial 
sources. This necessitates risk management program 
requirements and references directed to NDI PMs. 

The DODD 5000.1 contains six areas which could be revised 
to make its risk management guidance suitable for NDI PMs. 
The first area is in the risk management section description 


in Part 1. It presently states: 


Program risks and risk management plans shall be 
explicitly assessed at each milestone decision 
point prior to granting approval to proceed into 
the next acquisition phase. 


This guidance lacks reference to a definitive risk 
management planning structure. Currently, NDI PMs are 
expected to tailor the DODD 5000.1 risk management guidance to 
their NDI systems acquisitions. This leaves the PM’s 
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interpretations of present policy vulnerable to second- 
guessing by auditors and program stakeholders such as the 
Army . The following could be added for more structural 
content to developmental and NDI risk management planning: 


Every milestone decision point will include a 
review of the updated risk management plan (RMP) 
and measures taken to identify, assess, analyze, 
handle, and document program risks. Sone imuntey 
will be maintained between the RMP and discussions 
of program risks in the Integrated Program Summary, 
Acquisition Plan, Systems Engineering Master Plan, 
Test and Evaluation Master Plan, and Integrated 
Logistics Support Plan. 


The RMP would be modelled after the DSMC Risk Management 
and/or DSMC Systems Engineering Management Guides’ formats, 
depicted in the DOD 5000.2-M, and tailored to the requirements 
of the individual program. It would require the "teaming" of 
the user and the PM to integrate a risk management focus with 
needs and requirements development. 

The second area is subsection a. which describes critical 
parameters. This statement ignores the cost drivers in NDI 
programs: requirements, NDI acquisition management, T&E, S&S, 
and ILS. It directs the following: 


a. Critical parameters that are design cost 
drivers or have significant impact on readiness, 
capability, and life cycle costs must be identified 
early and managed intensively. 


The DOD 4245.7-M describes the areas of risk that 
jeopardize successful cost objective achievement. It does not 
describe NDI risks acting as cost drivers. The following 
could be added to reflect these NDI risks: 


@ 


NDI programs shall include the effects of 
requirements, NDI acquisition management, Test and 


Evaluation, Specifications and Standards, and 
Integrated Logistics Support risks on cost 
objectives. Measures of effectiveness and 
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performence for risk handling shall be proposed in 
the RMP concerning these risks. 


The third area of interest is subsection c. which 


discusses Test and Evaluation. It reads: 


c. Test and Evaluation shall be used to determine 
system maturity and identify areas of technical 
risk. 


NDI acquisition strategies often include accelerated and 
streamlined (depending on the quantity and quality of 
contractor test data) testing and evaluation processes or 
simply contractor test and evaluation data reviews. The risks 
in NDI acquisitions are teSting too little, too much, and the 
right or wrong requirements. The following should be included 


to address these concerns: 


NDI market analysis of proposed designs, NDI test 
and evaluation programs, and NDI performance 
specifications shall not be subject to trade-offs 
for the sake of short term cost and schedule 
objectives. Such trade-offs introduce risks to NDI 
lifecycle costs and support. NDI contractors’ test 
and evaluation data shall be screened and validated 
independently by the .operational test and 
evaluation community. 


The fourth area is subsection d. which considers 
contractor reSponsibilities. It directs that: 


ae Solicitation documents shall require 
contractors to identify risks and specific plans to 
assess and eliminate risks or reduce them to 
acceptable levels. 


The reference to eliminate risks could be removed since 
no program objective or action is free of risk. The shortfall 
in this guidance on contractor risk management is the lack of 
parallelism between the NDI PM’s risk management efforts and 
requirements and those of the contractor. The NDI risk 
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management efforts should emphasize teaming and sharing to 
relieve the burden of risk management from resting primarily 
on the Government. This is particularly true with the 
accelerated nature of NDI acquisitions. Risk responsibility 
Should be shared by the contractor. The following could be 
added: 


Contractor RMPs shall be specified in solicitations 
as a deliverable. These shall be consistent with 
DOD program managers RMPs. Risk sharing shall be 
emphasized by the DOD-contractor team. 


The fifth area focuses on subsection e. which discusses 
the risk assessment format found in DOD 5000.2-M. The 
guidance and format make no mention of NDI risks. Risk 
assessments, while intended to be tailored, should be 
comprehensive in nature and include the full spectrum of risks 
expected as a NDI PM. The five NDI risks should be added to 
the DOD 5000.2-M format. 


The final area of revision is subsection £f. which reads: 


fens Schedule shall be subject to trade-off as a 
means of keeping risk at acceptable levels. 


The FAAD Sensors case demonstrates that trading off 
schedule to control risk implies accepting force readiness and 
doctrinal capability gaps. If no operationally effective 
replacement systems exist, then such trade-offs can have 
Significant impacts on force training and combat 


effectiveness. The following clarification could be included: 


Such trade-offs shall be made only in the event 
that current systems exist to sustain the force 
until the NDI are fielded. 


The DODD 5000.1 risk management policy revisions are an 
important first step. Policy must clearly state the risk 
management requirements and guidelines as they specifically 
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affect a NDI PM’s risk management program. These changes will 
provide impetus for revisions to DODI 5000.2 which will be 


examined next. 
2. DODI 5000.2 Risk Management Policy 


NDI PMs need more definitive and explicit policy and 
procedures in DODI 5000.2, Part 5B pertaining to their risk 
management programs. The DODI 5000.2, with its developmental 
item orientation, neglects NDI risk management. With 
increasing competition for shrinking DOD RD&A budgets and 
resources, DOD and Congressional risk tolerance will decline, 
pressure to manage programs according to sound business 
practices will intensify, and "safer" acquisitions will 
attract more support. In these circumstances, explicit NDI 
risk management instruction will become critical to NDI 
program management. 

The two subsections in DODI 5000.2 could provide more 
definitive risk management guidance to NDI PM’s and remove the 
"Quesswork" required in tailoring developmental risk 
management policy. The first area is subsection a. No 
mention is made of supportability and programmatic risks in 
the reference to risk identification and control. These 
objectives typically encounter many risks in NDI acquisitions 
from risk events concerning requirements, NDI acquisition 
Management, T&E, and ILS. Supportability and programmatic 
risks should be included in the discussion of acquisition 
program risks. 

The DOD 4245.7-M templates referred to in the policy as 
a means of identifying and controlling risks are ten years old 
and list risk areas such as funding that are not matched or 
consistent with DSMC’s five objective areas of risk. The 
templates Nave no NDI-spee1tterconteaus The five areasmes 


NDI risks should be explicit in the templates so that they 
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correspond to NDI risk guidance in DODI 5000.2. DOD 4245.7-M 
revisions will be discussed in the next section. 

Subsection b. of DODI 5000.2 addresses industry 
participation in risk management. Currently, there is no 
explicit integration of contractor and PM risk management 
efforts. A contractor RMP deliverable similar to the plan 
published by NDI PMs binds the contractor to a risk management 
partnership with the PM. It should document and update how 
the program’s risk identification, assessment, analysis, and 
handling processes would be conducted. This uniform standard 
would entail closer coordination or "teaming" of risk 
management actions. 


3. DODI 5000.2 Risk Management Procedures 


Three areas exiSt in DODI 5000.2 risk management 
implementation procedures that could better serve the 
interests of NDI PM’s with more explicit discussion of NDI 
risks. First, the instruction, "include clearly defined 
criteria for elements leading to the risk assessment events... 
Satisfaction of these criteria must be well documented to 
Support the rigor necessary in the risk assessment process" 
highlights the policy void NDI PMs face in justifying their 
risk ratings and assessments. DOD 4245.7-M serves as the 
Origin for the criteria described. Without existing NDI risk 
criteria, risk assessments have no basis or references to 
Support them. 

The second area of this procedures section concerns the 
risk assessment format in DOD 5000.2-M. The format 
description does more to encourage tailoring than to depict a 
specific risk assessment format; it provides little to support 
outlining an NDI risk assessment. The format would be more 
consistent if it were to correspond to the risk areas listed 
in the DOD 4245.7-M. Additionally, with NDI and streamlining 
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initiatives more common today, more detail is required in the 
Supportability and programmatic sources of risk currently 
described in those sets of templates. 

Finally, milestone decision point reviews discussed in 
this portion of DODI 5000.2 do not refer to one central 
document as the basis for risk management data and 
documentation of the planned risk management program. 
Reference should be made to a RMP as the source of documented 
program risks, risk assessments and analysis, risk reduction 
measures, rationale, and assumptions in the published risk 
ratings. This could be more effective than flipping between 
the IPS, AP, TEMP, and SEMP. 

Alternative acquisition strategies, particularly NDI 
acquisition strategies, are often designed to mitigate certain 
risks. NDI acquisition strategies account for degrees of risk 
Since they are based on procuring mature technology with 
abbreviated RD&A schedules and costs. The policy here should 
explicitly cite the NDI alternative, its streamlined 
acquisition cycle, and the parts of DODD 5000.1 and DODI 
5000.2 in which detailed descriptions of NDI risks and risk 
Management actions appear. For example, the following parts 
of DODI 5000.2 could feature the NDI risks, possible risk 
ratings and assessment considerations, and initial risk 
handling options as deliverables: Part 4 (Requirements), Part 
6 (Test and Evaluation), Part 7 (Integrated Logistics 
Support), Part 9 (Specifications and Standards), and Part 11 
(NDI Acquisition Management). Risks, particularly NDI risks, 
could then be distinguished from developmental features in 
each of these parts of the DODI 5000.2. 


F. DOD 4245.7-M TEMPLATES IMPROVEMENTS 


The following are areas for revision in DOD’s primary 
risk management tool, the risk templates. This would clarify 
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DOD’s position on NDI risks and recommended risk management 
measures to the benefit of NDI PMs, their programs, and their 


customers. 
1. Requirements Risks 


NDI risk management templates should begin with 
requirements risk. Requirements risk should be identified and 
assessed first, prior to considering a program’s risks to its 
cost, schedule, technical performance, supportability, and 
programmatic objectives. Identifying and assessing 
requirements risks controls its potential impact on technical 
performance, cost, schedule, and performance specifications 
from stalling an NDI program’s early progress. NDI 
requirements must include integration and interoperability of 
the NDI with developmental items and the corresponding risks 
of failing to effectively integrate and interface. An example 
would be the risk of LSDIS not having interoperability with 
FAAD command and control nodes and fire units such as Avenger 
because that requirement was not concretely established. 

Inserting market analysis into the acquisition cycle and 
Screening requirements development against what actually 
exists in the market would limit unrealistic requirements 
appearing in RFPs. The market analysis step should be 
specified in policy as a step initially occurring between 
preparation of the MNS and the ORD. This would facilitate NDI 
solutions to user requirements and support sound NDI 
acquisition strategy development based on a clear picture of 
Current industry technology rather than nonexistent, desired 
Capabilities of multiple systems blended together. The 
category of NDI would also be clearly identified such that 
Source selection evaluation test results concerning assessment 
of technical performance would not necessitate redesign and 


recompetition of requirements and potential designs. 
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2. Requirements Risk Template 


AREAS OF RISK: 

Operational and Design Requirements that are ill-defined 
or overlook NDI alternatives. 

Inadequate Market Analysis that contributes to "mix and 
match technologies being required rather than verification of 
what technology actually exists. 

Tradeoffs in minimum and objective performance 
requirements that fail to meet the user’s stated need. 

Improperly defining the proposed system’s NDI category 
and required design modifications. 


OUTLINE FOR REDUCING RISK: 


1. Insert NDI Market Analysis into the acquisition cycle 
as part of Phase 0, Concept Exploration and 
Development. 


2. Apply PM - user teaming to better screen and develop 
requirements. 


TIMELINE: All Phases 
3. Specifications and Standards Risks 


NDI risk management policy should emphasize which 
specifications and standards, “CIDs or MILSPECS, that 
procurements will comply with. Secretary of Defense William 
Perry is on record as requiring the use of commercial 
specifications and standards rather than MILSPECS and MILSTDS 
(Secretary of Defense Memorandum on Specifications and 
Standards; =p. 1 1994 


Use performance and commercial specifications 
and standards in lieu of military specifications 
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and standards, unless no practical alternative 
exists to meet the user’s needs. 


The DODI 5000.2 clearly states the waiver required to use 
MILSPECS and MILSTDS over CIDs found in the Uniform Commercial 
Code. DOD policy on technical risk reduction by using mature 
NDI systems and their corresponding mature commercial 
specifications and standards can assist PM’s and simplify 
contractor compliance. Rather than specifications that 
specify system, design, and manufacturing functions and steps, 
the use of performance specs based on form, fit, and function 
in place of the current "how-to" specs better suits mature 
technology found in NDI. Such a change would facilitate 
contractor responsiveness to RFP requirements and draw more 
competitors to the bidding process. A larger sample of 
industry alternatives, as witnessed in the re-competition of 
the GBS contract and validated in SSETs, encourages 
competitive systems and prices. Using CIDs would replace the 
red tape NDI systems face in complying with developmental 
MILSPECS and MILSTDS. 


4. Specifications and Standards Risk Template 


AREAS OF RISK: 

Technical performance in commercial applications as 
Specified in CIDs may not equate to or explicitly meet 
technical performance in military applications as stated in 
MILSPECS and MILSTDS after an NDI acquisition strategy is 
already approved. 

Performance S&S based on form, fit, and function (that 
allow contractors to design solutions) instead of the "how-to" 
MILSPECS and MILSTDS used in design and manufacturing may 
encounter workforce resistance or complacency. 


Inadequate market analysis leads to accepting products 
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with insufficient or undocumented technical data or CIDs with 
which to re-compete the procurement for future buys. 
OUTLINE FOR REDUCING RISKS: 


1. Specify in solicitations that CIDs for meeting user 
requirements are a deliverable. 


2. Evaluate the CIDs against MILSPECS to verify their 
adequacy for design and development. 


3. Train and educate the acquisition workforce in CIDs 
and commercial specifications. 


TIMELINE: All Phases 
5. NDI Acquisition Management 


NDI acquisition management risks impact programmatic, 
cost, and schedule objectives. NDI risk management policy 
should redirect risk management responsibilities to be borne 
primarily by industry, or at least equally shared. Since NDI 
is mature technology, the developmental program risks assumed 
by the PM and the Government no longer apply. With CIDs on 
hand and effective market analysis verifying qualified 
contract bidders, the RFP and SSET processes become much 
Simplified. 

Changing developmental program paradigms to NDI remains 
an acquisition community challenge. NDI alternatives can 
become more acceptable to the acquisition workforce with 
revised risk templates and detailed articulation of NDI risks. 
Streamlining risk management paperwork requirements would be 
best instituted by using RMPs and improved education. iy 
addition, delegating reviews of program risks to lower levels 


of the acquisition chain of command would speed risk 
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management actions. 

The integration of civilian technologies into the 
military demands that contractor risk management programs and 
plans be explicitly required items in RFPs. NDI acquisition 
management requirements mandates parallel NDI risk management 


policy and references for both industry and PMs. 
6. NDI Acquisition Management Risk Template 


AREAS OF RISK: 

Despite streamlining, paperwork requirements, pricing 
data, accounting requirements and continuous audits of NDI 
programs stifle the cost and schedule objectives laid out in 
the acquisition strategy. 

Traditional developmental program attitudes and 
developmental program "mindsets" continue to reflect a 
cultural resistance toward implementing timely and cost- 
effective NDI acquisition strategies. 

Lack of PM-contractor and user-PM teaming on risk 
responsibility and risk sharing hampers flexibility in risk 
Management efforts. 

Programmatic micromanagement by stakeholders defeats the 
benefits of an NDI acquisition strategy. 


OUTLINE FOR REDUCING RISKS: 


1. Require RMPs as a contractor deliverable. 


2. Require workforce training and education in NDI 
through DSMC. : 


TIMELINE: All Phases 


Jal 


7. Test and Evaluation 


NDI Test and Evaluation risks impact performance, cost, 
schedule, and supportability objectives. NDI risk management 
policy must address the accelerated T&E program for validating 
component and system performance in operational testing. An 
NDI evaluation would be expedited with a test-fix-retest 
(FAADS PMO, 1994) cycle rather than the extensive 
documentation that goes into describing all the system faults 
prior to a rescheduling a test. Technical performance and 
design risks as well as confirmation of compliance with 
realistic specifications and standards, requires more detailed 
testing of the product and production process than implied in 
the NDI mature technology "label". 


8. Test and Evaluation Risk Template 


AREAS OF RISK: 

Requirements are not stable, realistic, or well- 
understood by designers, developers, testers, or managers. 

Overtesting conducted despite the presence of 
Satisfactory contractor test and evaluation data package. 

Developmental/technical testing costs are saved but 
operational testing for operational effectiveness and 
Suitability may involve conditions not grasped by contractor 
testing program. These incomplete tests and/or data may be 
overlooked or unquestioned in the accelerated NDI acquisition 


cycle and corresponding accelerated NDI testing program. 


OUTLINE FOR REDUCING RISKS: 


1. Test and evaluation data reviews of contractor 
commercial testing program and results. 
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peweemonstrations of the contractor’s testing process. 


3. Test-Fix-Retest the proposed design on site to 
expedite testing. 


TIMELINE: All Phases 
9. Integrated Logistics Support 


NDI ILS risks affect cost, schedule, performance, and 
Supportability objectives. NDI risk management policy on 
logistics planning risks entails the greatest need in improved 
DOD guidance. The supportability risk of contractor 
production and support operations terminating prior to a time 
convenient to the Government, mandates that options be 
specified in the ILSP and agreed upon by the PM and the 
contractor. 

Options the PM can consider include those posed in DSMC’s 
NDI Acquisition publication: 

a. Buy commercial upgrades as they evolve and 
become available. 

b. Make a one-time mass spares purchase to sustain 
the duration of the system’s lifecycle. 

om Buy the technical data package to solicit 
sources of supply that coincide with the end of original 
production and support by the original contractor. 

Market analysis should reveal whether contractor 
technical data rights conflict with Government interests and 
whether longevity of profitable production lines pose risks to 
achieving supportability objectives. These should all be 
meints Of direction in DOD policy. 
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10. Integrated Logistics Support Risk Template 


APEBAS OF RISK: 

Technical Data Packages may be unavailable or incomplete 
which creates instability of spares and parts access. 

Competitive re-procurements of parts may not contain 
proper incentives to attract spares and parts vendors. 

ILS and lifecycle focus may be overlooked during the 
requirements development stage. 

Use of military standard and non-standard parts creates 
multiple parts and spares lines. 

Depot/repair levels may not be defined in terms of 
operational environments. 

OUTLINE FOR REDUCING RISKS: 


1. Define ILS requirements when deciding what category of 
NDI the acquisition strategy involves. 


2. COnduce market analysis ene contractor ILS 
capabilities, ILS testing and support demonstrations 
in the intended operational environment and 
COndle1ens: 


3. Specify training packages and publications as a 
contractor deliverable. 


G. SUMMARY 


The NDI areas described above require timely DOD 
attention in DODD 5000.1, DODI 5000.2, and DOD 4245.7-M. 
These revisions are needed to adequately provide guidance to 
NDI PMs. The template diagram, .the specific area of risk, an 
outline for reducing the risk, and a timeline for managing 


these risks within the system’s lifecycle must be included. 
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Clarifying the risk management policy and corresponding 
templates with NDI implications would streamline the risk 
management process in itself by saving time and effort to 
staff who manage a risk management program. There would be no 


need for NDI interpretation of developmental item templates. 


g5 
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VI. SUMMARY, CONCLUSIONS, AND RECOMMENDATIONS 
A. SUMMARY 


The subjects of risk management and NDI acquisitions are 
attracting more attention today. The individual emphasis on 
these two acquisition management areas is not matched by their 
effective integration. As a result, NDI PMs are conducting 
risk management programs that are themselves at risk due to 
Miemiacx Of current, explicit guidance from DOD. 

The practice of risk management does not benefit from 
meooroo00k" solutions. If such solutions existed for 
developmental acquisition programs, there are very few, if 
any, for NDI acquisition programs. This is the NDI PM’s 
dilemma. The NDI PMs must adhere to DODD 5000.1 and DODI 
5000.2 risk management policies in their programs, but the DOD 
material is vague or must be tailored to support NDI program 
needs. The FAAD Sensors case is a testament to the PM who 
drives a risk management program without the benefit of 
Signposts or road maps. 

The NDI PM’s challenges and program risks in the mid- 
1990's are not adequately reflected in DOD’s mid-1980’s risk 
Management policy, procedures, or tools. Risk management will 
receive more, not less, emphasis as an explicit management 
function. NDI and NDI acquisition strategies will continue to 
grow in popular support as DOD RD&A budgets "downsize". 
Modernization at minimum risk therefore will require properly 
marked signposts and a good road map. It is time for DOD to 


print and distribute those signs and maps. 
B. CONCLUSIONS 


NDI PMs continuously manage risk as part of today’s 
streamlined and tailored NDI acquisition environment. They 
Should expect and receive succinct, explicit policy and 
guidelines to help them meet their risk management and program 
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management goals and objectives. DODD 5000.1 could better 
serve the NDI PMs with risk management language directed to 
their specific type of programs and acquisition strategies. 
DODI 5000.2 could provide NDI PMs with both better risk 
management guidance and implementation procedures with NDI- 
based instructions, formats (to be found in DOD 5000.2-M), and 
tools (DOD 4245.7-M risk templates and risk management plan). 

DOD 4245.7-M could introduce NDI as a topic and an area 
of templating as it is updated for today’s acquisition 
environment. The risks uniquely significant to NDI 
acquisition programs can be better recognized, understood, and 
managed with a focal point known as a risk management plan. 

Several common threads bind NDI and NDI risk management 
to immediate programmatic, performance, and supportability 
objectives and long-term cost and schedule objectives. Those 
threads are thorough market analysis, accelerated acquisition 
cycles, and teaming of the risk management effort between the 
PM and the contractor as well as the PM and the user. The 
acquisition environment and its intricacies may be changing 
but the demand for short-duration, cost-effective systems 


acquisitions will persevere. 
C. RECOMMENDATIONS 


DOD should upgrade the DODD 5000.1, DODI 5000.2, DOD 
4245.7-M, and DOD 5000.2-M to incorporate the NDI risk 
management issues and proposed revisions discussed in this 
thesis. DSMC publications referring to risk management and 
the AFAM should be similarly revised. 

Market analysis, risk management plans, teaming, and NDI 
acquisition training and education should be explicitly 
emphasized in DOD risk management policy. Market analysis 
Should be specified as a deliberate step or phase conducted 
immediately after the release of the Mission Needs Statement. 
This may coincide with the Concept Exploration and Development 


ens) 


Phase. Risk management plans should be a required program 
management and decisionmaking tool that is published as the 
primary source of risk management program information such as: 
risks, risk ratings, risk assessments, risk analysis, risk 
handling and/or reduction measures, and risk data 
documentation. Other program plans would refer to the RMP as 
the consolidated reference point for program risk information. 
Teaming and education on NDI risk management should be 
directives required of all PMOs. 

Risks in the following NDI sources of risk should be 
explicitly cited and referenced in DOD risk management policy: 
Requirements, Test and Evaluation, Specifications and 
Standards, Integrated Logistics Support, and the NDI 
Acquisition Management Environment. These sources create 
uncertainty in the successful accomplishment of NDI program 
objectives. They should be explored from both a NDI and a 
developmental perspective (based on the needs and requirements 
Of the user). 

DOD NDI risk management policy should stress RMPs, CIDs 
(for review and comparison to MILSPECS), Test and Evaluation 
program data packages and demonstrations, and Integrated 
Logistics Support Plans in NDI RFPS as_ contractor 
deliverables. These should be required up front and early in 
the accelerated NDI acquisition cycle and subject to frequent 
review based on their immediate and long-term lifecycle 


impacts on the system being acquired. 


D. AREAS FOR FURTHER RESEARCH 


1. Market Analysis of NDI Requirements. The importance 
of market analysis grows as DOD turns to the commercial 


marketplace for more immediate solutions to. its needs. An 
analytical approach to evaluating NDI solutions is needed to 
replace the market surveys of the past. This is more relevant 


given the close ties and tremendous impacts well- or poorly- 


Sle) 


done market analysis can have on requirements generation, 
selection of specifications and standards based on those 
requirements, and comparison of contractor test and evaluation 
results with those sought through DOD’s developmental and 
operational testing programs. Market analysis also plays a 
Significant role in determining contractors’ ability to 
Support their products throughout their lifecycles. A case 
study of NDI market analysis success story would prove 
valuable as a reference or model to all acquisition community 
players. 

2. Successful Contractor RMPs (in the Context of Best 
Practices). The case analysis of one or several success 
stories in contractor risk management bears investigation, 
particularly in the NDI environment. The lack of DOD emphasis 
on risk management of NDI acquisition programs and supporting 
RMPs may be mirrored by a lack of commercial emphasis on the 
same area. Documentation of lessons learned in contractor 
risk management plans could offer valuable insights to the 
user community, PMs, and other contractors. Providing a 
contractor "best practices risk management model" could 
reinforce the importance placed on contractor RMPs as 
deliverables. 


3. Risk Management in Bose sci Military Sales and 


Foreign Military NDI acquisition programs. The risks 
encountered with acquiring NDI may represent only one category 


of emerging acquisition issues facing DOD. Risks and risk 
management practices in foreign procurements may have 
Similarities to NDI acquisition programs. This is especially 
true with our national security strategy’s increased focus on 
economic security and global competitiveness. The DTIB’s 
vulnerability to not only national NDI alternatives but 
international NDI may provide more justification for dual-use 
products to improve competitiveness and mitigate RD&A risks. 


The risks and risk management practiced in foreign acquisition 
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programs will become more complex and may require more 
detailed DOD risk management policy in foreign technology 
acquisition programs. 

4. The DOD Developmental Acquisition Program Community’s 
Ability to Cope with the Rising Tide of NDI and Commercial 
Practices. The cultural biases and mindsets towards 
developmental programs described in this thesis must face the 
rising tide of NDI much like the DTIB faces increased 
competition for shrinking RD&A dollars. Changing attitudes 
and program management paradigms in the acquisition community 
has tremendous time, resource, and personnel implications. 
Streamlining and tailoring the DOD acquisition process 
emphasizes immediate, mature, and cost-effective NDI 
PeeerMatives. It may be important to examine the training and 
education programs being used to facilitate NDI acquisitions, 
NDI risk management, and acquisition community responsiveness 


to a changing acquisition environment. 
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